Consumer electronic registration, control and support concierge device and method

ABSTRACT

We disclose a concierge device that can be configured to register, control and support a consumer device. It can alternatively or redundantly connect to a home management bridge and/or cloud-based management servers. It can accept menus that allow a single concierge device to provide a wide range of functions for various consumer devices. The concierge device allows the user in a single action to initiate a support session, automatically identifying the consumer device. The concierge device can be configured for voice or video support calls. The concierge device in conjunction with a home management bridge or gateway can manage on boarding of components of an automated home, such as switches and lamps. Implementations of the concierge device that include a display can show supplemental information, such as advertising, optionally in coordination with media being played on a consumer device coupled in communication with the concierge device.

RELATED APPLICATIONS

This application is a Continuation of U.S. application Ser. No.14/281,781 filed 19 May 2014, entitled “Consumer ElectronicRegistration, Control and Support Concierge Device and Method, now U.S.Pat. No. 9,866,697, issued 9 Jan. 2018 (NEXS 1010-10), which is aContinuation of U.S. application Ser. No. 13/344,529 filed 5 Jan. 2012,entitled “Consumer Electronic Registration, Control and SupportConcierge Device and Method”, now U.S. Pat. No. 9,614,964, issued 4 Apr.2017 (NEXS 1010-5); which claims the benefit of the followingprovisional applications: U.S. Prov. App. No. 61/561,732 filed 18 Nov.2011, entitled “Consumer Electronic Registration, Control and SupportConcierge Device and Method” (NEXS 1010-2); U.S. Prov. App. No.61/530,370 filed 1 Sep. 2011, entitled “Consumer ElectronicRegistration, Control and Support Concierge Device and Method” (NEXS1010-1); U.S. Prov. App. No. 61/430,099 filed 5 Jan. 2011, entitled“Consumer Electronic Support Concierge Device and Method” (NEXS 1007-1);and U.S. Prov. App. No. 61/430,103 filed 5 Jan. 2011, entitled“Advertising on a Tethered Digital Butler Consumer Electronic Device andMethod” (NEXS 1008-1). These priority applications are herebyincorporated by reference.

This application is related to and generally consonant with U.S. Pat.No. 8,243,886, based on application Ser. No. 13/345,447 (NEXS 1010-4)filed the same day as U.S. application Ser. No. 13/344,529.

This application is further cross-references but does not claim priorityto U.S. application Ser. No. 13/047,445 filed 14 Mar. 2011, entitled“Tethered Digital Butler Consumer Electronic Remote Control Device andMethod”, now U.S. Pat. No. 8,494,132, issued 23 Jul. 2013 (NEXS1000-15); which is a Continuation of U.S. application Ser. No.12/758,241 filed on 12 Apr. 2010 entitled “Tethered Digital ButlerConsumer Electronic Remote Control Device And Method”, now U.S. Pat. No.7,907,710 (NEXS 1000-13); which is in turn a Continuation of U.S.application Ser. No. 12/572,677 filed on 2 Oct. 2009, entitled “TetheredDigital Butler Consumer Electronic Remote Control Device And Method”(NEXS 1000-11), now U.S. Pat. No. 7,697,669; which is in turn aContinuation of U.S. application Ser. No. 12/420,723 filed on 8 Apr.2009, entitled “Tethered Digital Butler Consumer Electronic Device AndMethod” (NEXS 1000-9), now U.S. Pat. No. 7,609,824; which is in turn aContinuation of U.S. application Ser. No. 12/210,782 filed on 15 Sep.2008, entitled “Tethered Digital Butler Consumer Electronic Device AndMethod” (NEXS 1000-8), now U.S. Pat. No. 7,542,753; which is in turn aContinuation of U.S. application Ser. No. 11/465,749 filed on 18 Aug.2006, entitled “Tethered Digital Butler Consumer Electronic Device AndMethod” (NEXS 1000-3), now U.S. Pat. No. 7,444,130; which is in turn aContinuation-in-Part U.S. application Ser. No. 11/350,980 filed on 8Feb. 2006, entitled “Tethered Digital Butler Consumer Electronic DeviceAnd Method” (NEXS 1000-2), now U.S. Pat. No. 7,389,103; which claims thebenefit under 35 USC 119(e) of U.S. Prov. App. No. 60/709,666 filed on19 Aug. 2005, entitled “Tethered Digital Butler Consumer ElectronicDevice And Method” (NEXS 1000-1).

BACKGROUND

The field of this disclosure for selection of an Art Unit to examinethis application includes systems, processes and devices for remotecontrol, in communication with consumer devices such as lighting,entertainment appliances, heating, ventilation, air conditioning,laundry, and kitchen appliances. The systems, processes and devicescommunicate with cloud-based resources via a telephone line (e.g., DSL)or other two-way electromagnetic or optical transmission. One feature ofsome implementations is enabling conversation; another is transmissionof speech or the like.

The home of the future has captured our imagination for decades andprobably for centuries. For instance, Disneyland hosted a home of thefuture from 1957 to 1967, as a joint project of Monsanto, MIT andDisney. In that era, resources were plentiful and computers wereexpensive corporate devices.

Recently, IBM has published “The IBM vision of a smarter home enabled bycloud technology” (September 2010), which offers a much different visionof the home of the future, one supported by evolving cloud-basedanalytics. This white paper, which describes IBM's vision of a connectedhome and of the “Internet of things,” accompanied one or more of theprovisional applications and is hereby incorporated by reference.

The evolving vision of the home of the future creates an ongoingopportunity for innovation, for communication devices and homeaccessories that will improve our lives, control and support theinterconnection of our consumer devices with services outside our homes.For some purposes, one can distinguish between consumer appliances (CA)and consumer electronics (CE), distinguishing between appliances such asdishwashers, washing machines, heaters, air conditioners, lighting andthe like, versus CE television, entertainment and media center devices.New methods and devices may enhance the service experience of consumers,particularly users of leading consumer electrical and electronicdevices.

SUMMARY

We disclose a concierge device that can be configured to register,control and support a consumer device. In various implementations, italternatively or redundantly connects to a home management bridge and/orto cloud-based management servers. It can dynamically accept menus thatallow a single concierge device to provide a wide range of functionscustomized to particular consumer devices in a home. The conciergedevice, when associated with a consumer device, allows the user in asingle action to initiate a support call, automatically identify theconsumer device, and optionally either bypass automated attendant choicetrees or invoke an interactive voice response protocol or invoke avisual interaction protocol that takes into account information aboutthe consumer device in the home. The concierge device can be configuredfor voice or video support calls. The concierge device in conjunctionwith a home management bridge or gateway can manage on boarding ofcomponents of an automated home, including home control devices andcontrollable devices such as switches and lamps that are connectedwirelessly. Implementations of the concierge device that include adisplay can show supplemental information, such as advertising,optionally in coordination with media being played on a consumer devicecoupled in communication with the concierge device. Particular aspectsof the present invention are described in the claims, specification anddrawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a high level conceptual diagram that includes a conciergedevice, a variety of consumer devices in a home, a home managementbridge and a cloud-based system delivery platform connected by anetwork.

FIG. 2 depicts a concierge device in a variety of form factors.

FIG. 3 depicts telephone and support service as functions that might beselected using a concierge device.

FIGS. 4-8 show configurations of concierge device and home managementbridge, either integrated or standalone. FIG. 4 depicts a conciergedevice built into a refrigerator or other major appliance. FIGS. 5 and 6depict scenarios using a concierge device without a display. FIG. 7depicts a concierge device that includes a display, in use with a homemanagement bridge or alternatively connecting directly to cloud-basedmanagement servers. FIG. 8 depicts a concierge device with an integratedbridge and touch screen display.

FIG. 9 is a flowchart of methods consistent with this disclosure.

FIGS. 10A, 10B, 11A, 11B, 12A, 12B, 13A, 13B, 14A, 14B and 15 areexample screens for a concierge device that includes a display. In FIG.10A, users are presented with choices, including invoking the conciergeservice described. FIG. 10B depicts selection of TV channels from liveTV. FIGS. 11A-B depict a video phone call. FIG. 12A shows an example ofuser authentication using biometric data. FIG. 12B shows an example ofan alert that a new home control device, a switch is available foron-boarding. FIG. 13A is an example GUI that used to select among devicetypes to which a user can connect the new switch. FIG. 13B is an exampleGUI that presents devices that a user can connect to the new switch.FIG. 14A is an example of GUI from which a user can select a device andcause initiation of a support session. FIG. 14B is an example of asuggested fix presented to a user in a support session prior toconnection with a human agent. FIG. 15 depicts a video call with a humansupport agent.

FIG. 16 depicts information that may be useful for the home managementbridge or the concierge device to communicate to the support center.

FIG. 17 is a high level diagram of adding to support an automatedinteraction for the user.

FIG. 18 represents a sample of information that may be supplied as partof an automated interaction.

DETAILED DESCRIPTION

The following detailed description is made with reference to thefigures. Preferred embodiments are described to illustrate thetechnology disclosed, not to limit its scope, which is defined by theclaims. Those of ordinary skill in the art will recognize a variety ofequivalent variations on the description that follows.

Concierge Device Range of Capabilities

We disclose a concierge device that can be configured to register,control and/or support a consumer appliance or electronics device(collectively, “consumer devices”). It is particularly innovative toregister, control and/or support consumer appliance devices, asappliances have not traditionally been smart, have not been managed inthe home, and have not been connected to the cloud. The conciergedevice, in various implementations, may connect to a home managementbridge or to cloud-based management servers. In other implementations,it uses one as a back-up to the other.

In some implementations, the concierge device accepts consumer devicecontrol menus from either a home management bridge or cloud-basedservers that are appropriate to a registered consumer device. Thesedynamically accepted menus allow a single concierge device to provide awide range of functions customized to particular consumer devices in ahome. Optional authentication security can be implemented to satisfysecurity conscious home owners.

The concierge device also can, in some implementations, displaysupplemental information to the user, such as targeted or interactiveadvertising.

While this disclosure refers to a home, the technology disclosed canalso be applied in a business, such as a small business. Where referenceis made to “in-home” technology, this translates to “on premises” for abusiness.

Easy On Boarding Devices to Home Automation

Easy on boarding of a new device in the home can be part of aregistration process. This registration process can be made secure byauthentication. Any of the concierge device, home management bridge, orcloud-based service can recognize the presence of the new device in thehome network. The new device can communicate via a wireless or wiredprotocol as part of the home network. Optionally, the new device couldcommunicate with the concierge device and the concierge device couldrelay registration communications between the new device and the bridge,the cloud based service or both.

The concierge device can participate in registration and, optionally,authentication of the new device being on boarded. A concierge devicewith a speaker or other audio output can receive voice prompts forconfirming registration of the new device and can accept single useractions in response to the prompts, such as button presses, menuselections or verbal responses to the verbal prompts. Alternatively, aconcierge device with a display can receive visual prompts forconfirming registration of the new device and can accept single useractions in response to the prompts, such as button presses, menuselections or verbal responses to the visual prompts.

Authenticated On Boarding

Easy on boarding can be made secure by an authentication protocol. Oneauthentication protocol would rely on voice prints. The concierge devicepreferably would relay spoken words to the in home bridge or thecloud-based service and rely on the other device to use voice printingor another biometric analysis of the voice to authorize on boarding thenew device. Alternatively, the concierge device could, at a higher cost,analyze spoke words using resources built into the concierge device andperform local authentication from spoken words. Another authenticationprotocol would involve reading a user's fingerprint, using a fingerprintreader. The concierge device preferably would relay raw fingerprint datato the bridge or cloud based service. Alternatively, again at anincreased device cost, the concierge device could locally process thefingerprint data for authentication. Other challenge and responseauthentication protocols could easily be implemented using the conciergedevice in communication with the bridge or cloud based service or usinglocal resources of the concierge device. Possession of the conciergedevice combined with an additional authentication would providetwo-factor authentication.

At the simplest authentication level, the concierge device could bepaired with the bridge or cloud based service and a single press of abutton, shake of the device (sensed by an accelerometer, compass,gyroscope or other motion sensitive electronic, MEMS or other device) orspoken response to complete the on boarding. In this simple case,possession of the concierge device and a single user action would beenough to satisfy one factor authentication of the new device. Requiringsuch authentication inhibits new devices from registering themselves andbecoming viruses or Trojans on a home network.

In combination with either one or two factor authentication, the bridgeor cloud based service could describe the device being on boarded oradded to the home network and have the user confirm that the devicebeing added matches the description. The description could be eitheraudio or visual. This is an additional safeguard against mistaken ormalicious registration of new devices onto the network.

Among the functions that can be implemented, the concierge device mayrely on either the bridge or cloud-based services to transcode mediafrom other sources to a concierge-standard format, which reduces thecomplexity and cost of the concierge device. Optionally, the conciergedevice, when associated with a consumer device, allows the user in asingle action to initiate a support call without requiring the user tolook up or recall the support phone number, automatically identifying atleast the consumer device and optionally the user or household. Usingthe automatically provided information, the concierge device,optionally, could further allow a user to bypass automated attendantchoice trees, to invoke an interactive voice response protocol thattakes into account information about the consumer device in the home, toinvoke a session using a display of the concierge device (when itincludes a display), and/or to initiate a coordinated browser-basedsession involving a home computer. Examples of consumer devices includetelevisions, stereos, A/V receivers, satellite and cable receivers, DVDplayers, media servers, telephone systems, dishwashers, ovens,refrigerators, other kitchen appliances, washing machines, dryers, spas,HVAC, pool equipment, lighting, security sensors, medical devices,vehicles, utilities, personal computers, and network attachedappliances.

A concierge device, in some implementations, alternatively connectsthrough redundant devices, which reduces the chance of consumerfrustration with a smart home that needs to be rebooted or that hasinaccessible functions, due to management device malfunctions orinaccessibility. By redundant devices, we mean a home management bridgeand one or more cloud-based management servers. Examples of homemanagement bridges include those under development (at the time of thisapplication) by Ionics EMS and 4Home. These devices are sometimesreferred to as gateways. Being cloud-based, the management servers areaccessed via the Internet or another network such as a telephone,cellular, cable or fiber optic connection to a home network. By server,we mean a computing device that can be a workstation, server, servercluster, server farm or the like. The server may be an array of serversconnected to a load-balancing device.

The concierge device is wirelessly coupled to the home management bridgeand/or the cloud-based management servers. In some environments, it iscoupled through an access point, which is connected in turn to both theInternet and an in-home network. Many other configurations are possible,consistent with this disclosure, that also afford alternativeconnections from the concierge device to the home management bridgeand/or the cloud-based management servers.

The concierge device's first choice between alternative connections canbe the in-home device, as latency on a local network should be less thanlatency in connecting to a cloud-based service. The in-home device canstore information that the user does not want to have leave the home.The in-home device can obtain information as necessary from thecloud-based servers, connecting with them via the Internet or anothernetwork. For cloud-based services, a telecom or cable company couldrespond to IP messages from a concierge device without the messages everleaving the company's network, without a hand-off from the originatingnetwork to a network owned by a second entity. Both connection via thenetwork and a connection hosted by or affiliated with a last mileservice provider are part of what we refer to as cloud-based.

Alternatively, the concierge device may first attempt to connect to thecloud-based service and then fall back to an in-home bridge as analternative. The fall back may result from unavailability of orexcessive latency in connecting to the cloud-based home managementserver.

The home management bridge, in one implementation, mediates registrationand deployment of menus to the concierge device. The concierge deviceconnects to the home management bridge to register an associatedconsumer device. The operation of the home management bridge istransparent to the concierge device. The home management bridge mayalready have cached or permanently installed information for thespecific associated consumer device, for instance cached from the cloud.Or, it may have manufacturer connection information that allows thebridge to resolve a particular model number of the associated consumerdevice, register that type of device, retrieve control menus andretrieve other information. Alternatively, the bridge may rely oninformation initially stored by the associated consumer device. Thisinformation may be forwarded by the concierge device or obtained by thebridge directly.

In another implementation, the concierge device and associated consumerdevice begins by interacting with the cloud-based home management serverand the role of the home management bridge is to back up informationsolicited by the concierge device or associated consumer device from thecloud-based home management server.

Virtual Apps—Downloaded Menus

Following registration either with, through or bypassing any in-homebridge, the concierge device receives and optionally stores menus and/orcontrol codes for the registered consumer devices. Periodically, theconcierge device may check for updated menus and/or control codes. Itcan check with the home management bridge or with cloud-based servers.The home management bridge may periodically refresh a cache of menus andcontrol codes for devices registered by the concierge devices and otherin-home devices that it can detect.

A concierge device can obtain from a home management bridge menu,control code and other information, even if the concierge device was notinvolved in registration of in-home consumer devices. For instance, areplacement or second concierge device can be reloaded by the homemanagement bridge with information previously used by a first conciergedevice. Or, a first concierge device can be introduced into a home inwhich the bridge already had discovered consumer devices and downloadedappropriate information from cloud-based management servers.

One use of the concierge device is as a remote control. Another is tocontrol display of information buffered by the home management bridge ona television or other available on the same network as the bridge. Afurther use of the concierge device is to provide support for theconsumer device.

Concierge Device for One Action Support

Four levels of support automation are described. At the basic level, theconcierge device provides telephone support, following automaticidentification of user/household and/or consumer device to the supportcenter by either the concierge device or a home management bridge.

At the next level, the concierge device includes a camera that providesstill or video information to a support center during the support call.Optionally, a personal computer, television or other device with adisplay can receive visual information to assist the user during thesupport call. The visual information may be an image of the supportperson, pictures illustrating aspects of the consumer device, textualinstructions or a combination thereof

Automated troubleshooting can be pursued before a user is connected witha human support agent, using either a visual based interaction or aninteractive voice system. The automated trouble shooting may requiremandatory responses from the user, or it may be optional: opt in byasking a user if they want to try automated trouble shooting; opt out bystarting the automated protocol and giving the user the option toproceed to human support without further automated support; or opt incombined with opt out. Alternatively, the automated troubleshooting mayhappen while waiting in queue to be connected to a human support agent.The troubleshooting may involve a traditional interactive voice responsesystem, a browser-based visual troubleshooting protocol, or an advancedinteractive system based on the Watson artificial intelligencetechnology demonstrated by IBM. Trying the automated trouble shootingmay be optional during the queue period. A support system may beconfigured to allow a user to postpone human support, if automatedsupport seems to be working, either by using the support interface or bytelling the human support agent to come back later. A user whose turn totalk to a human support agent comes up in queue while pursuing automatedtrouble shooting may be kept at the top of the queue, so that theyreceive assistance promptly after completing automated trouble shooting,to improve user reception of the automated trouble shooting option.

Instead of performing automated trouble shooting with a display, anautomated voice response (AVR) system could be used to collectinformation and lead the user through basic troubleshooting steps. Muchof what can be done with a display could, alternatively, be done usingvoice interactions with an automated system.

At the third level, the concierge device causes diagnostic statusinformation to be forwarded from the consumer device to the supportcenter. This level optionally includes the camera of the previous level.Alternatively, configuration of the concierge device with the consumerdevice and the home management bridge may enable the consumer deviceperiodically or occasionally to provide status and/or diagnosticinformation to a support center, without the need for the user toinitiate a support call. Then, when the user initiates a support call,the support center may already have the information needed to identifyproblems with the consumer device, not necessarily limited to theproblem that leads to the support call. Optionally, automatedtroubleshooting, described above, also can be pursued before a user isconnected with a human support agent. The availability of this featureand option, of course, depends on the status of the consumer device. Thedevice may be dead, diagnostically accessible, or working anddiagnostically accessible. By “dead,” we mean that there is noelectronic communication with the consumer device, either directly orindirectly. By diagnostically accessible, we mean that the conciergedevice or a bridge working with the concierge device is able tocommunicate with the consumer device, at least for diagnostic purposes.This may involve a special mode, such as the “safe boot mode” on apersonal computer. By working and diagnostically accessible, we meanthat the consumer device is largely functional, but requires support.The diagnostic information forwarded may include one or more statuscodes. It may include software/firmware version information andcomponent inspection results.

At the fourth level, the concierge device or a bridge working with theconcierge device relays communications between the consumer device andthe support center in a diagnostic session. This level optionallyincludes features of the prior levels and nicely builds upon them. Thediagnostic session may involve the support center triggering aself-diagnostic routine or selective retrieval of information helpful todiagnostics, such as a history of status/error codes or operatingstatus, configuration information, or a dump created when the consumerdevice malfunctioned. It also may involve sending information from thesupport center to the consumer device, such as new firmware, revisedconfiguration data or reset commands. The stream of communication ordiagnostic session status information optionally may be displayed orsummarized for the user to monitor on a personal computer, television orother device with the display. Even before a human support agent becomesinvolved, an automated support protocol could, optionally, be used toengage the user in the communications between the consumer device andthe support center diagnostics. A system could automate such routinetasks as obtaining the user's permission to engage in interactivediagnostics, obtaining user visual observations of the device,identifying procedures that the user already has followed, obtaininguser contact and/or registration information. A wide range ofinformation collection and diagnostics could be automated, not limitedto these examples.

The simplest concierge device lacks a display or includes a minimalmonochrome display. It need only have a microphone, speaker, one to afew function buttons or input sensors, a memory and at least onewireless transceiver. The simple concierge device could rely on a bridgeto transcode voice communications and, optionally, pictures. It alsorelies on the bridge to maintain communications with the support centerduring the support call. In operation, the simple concierge device isassociated with the consumer device and then the user initiates thesupport call for the associated consumer device with a single action.For instance, the user could press a support button or utter a commandthat the concierge device or the bridge would interpret. Then, the userproceeds with the support call.

The bridge can communicate directly with the Internet or a privatenetwork or may be attached through a router, bridge or other networkdevice. The bridge may include router and/or modem functionality, as isfound among cable and DSL modems. Preferably, malfunctioning of the homemanagement bridge will not interfere with connection to the Internet ora private network.

The concierge device, even in its simplest configuration, can supportmultiple devices and/or multiple brands of devices. Given thefunctionality and alternative protocols disclosed, almost anyconfiguration of concierge device could be used with a TV from onemanufacturer, a DVR from another, and a washing machine from a thirdmanufacturer. The one exception might be a crippled configurationshipped with a device and purposely limited to a single device or singlemanufacturer.

Associating (Linking) the Concierge Device and Consumer Devices

There are a variety of options for associating the simple conciergedevice with a particular consumer device for support calls. Oneassociation model is the familiar Bluetooth pairing model, in which theconsumer appliance or electronic device and simple concierge device areplaced in pairing mode and allowed to pair. Information is transferredfrom the consumer device to the concierge device during or following thepairing. Another option is for the in home bridge to discover consumerdevices and for the concierge device to pair with the bridge, withouthaving to pair with any of the consumer devices. In someimplementations, a consumer may find it more convenient to pair justonce with a bridge than to pair repeatedly with a wide variety ofdevices. Pairing with just one bridge would save the consumer fromrecognizing slight variations on pairing protocols that would inevitablyfollow from pairing with devices of all sizes, shapes and origins.

Information provided by the consumer device to the concierge device orbridge may be as simple as a serial number or it may include furtherinformation that identifies the device manufacturer and model number.Alternative identifiers may be globally unique user identifiers (GUIDs)or identifiers that are unique within a realm, such as unique to amanufacturer or a trade group. Optionally, it may identify the devicecategory and/or family so that appropriate support staff can be reached,even if the precise model number is not in a support database. Theconsumer device may have embedded a unique resource identifier (URI),such as a universal resource locator (URL), with or without additionalparameters, that identifies or allows lookup of the support center, asan alternative to identifying the manufacturer and model.Pre-installation inventory of control and/or controllable devicecomponents can be accomplished using camera or bar code reader to read abar code or similar label. Or, an RFID reader can be used to read a tag.Optionally, a digital floor plan could be used to ease the burden ofcommissioning or on boarding of devices.

Another association model uses the home management bridge to associatethe concierge device with the particular consumer device that needssupport. The home management bridge can be used to register one or moreconsumer devices with at least one support center. The bridge and thesimple concierge device are paired or at least coupled in communication.The user interacts with the bridge to select the particular consumerdevice that needs service. Then, a single action at the concierge deviceor bridge initiates the support call. The concierge device participatesin a support session at one of the support levels described above.

A third association model programs the concierge device with theconsumer device information without requiring direct communication withthe particular consumer device by either the home management bridge orthe concierge device. For instance, the concierge device can bepreprogrammed at the factory. Or, it can be programmed using the bridgeor any personal computer, television or other device with the display.For instance, it could be coupled using a wired USB connection or awireless Bluetooth connection to a personal computer, television or DVDplayer that runs a small application which loads the consumer deviceinformation into the concierge device.

Of course, other association models could be used and remain consistentwith the teachings of this disclosure.

Example of Support Session Implementation

At the outset of the support session, the bridge sends the supportcenter at least consumer device information and, preferably, useridentification information. The user identification information can beas simple as a globally unique identifier (GUID) of the conciergedevice, home management bridge or consumer. For the concierge device andhome management bridge, a MAC address may serve as a GUID. This GUID maybe registered with the support center before the support call orregistration information may be collected during the support call.Preferably, the user's name would be associated with the GUID. The useridentification information may be household related, individual related,or both. Either a household or individual can have more than oneconcierge device. Information about the consumer device or a pluralityof consumer devices may be implied in the GUID. Optionally, the GUID,the particular consumer device identification or a combination of theGUID and consumer device identification can be associated with a premiumsupport level, such as contract support or extended support. Premiumsupport association entitles the support call to special handling, suchas a shorter wait queue or different support personnel.

A display equipped concierge device adds a display to the simpleconcierge device. The display equipped concierge device can havenavigation controls or a touch sensitive screen. It can have a keyboardimplemented with buttons or on-screen. The display equipped conciergedevice can interact with the bridge and the support center withoutinvolvement of a personal computer television. The display equippedconcierge device can be used to register the consumer devices with thesupport center or to enter information that will be useful at the outsetof the support call. It also can be used to answer diagnostic questionsat the beginning of the support session, before a support person comeson the phone. The diagnostic questions may depend on the consumer deviceinformation supplied to the support center.

The display equipped concierge device can rely on the home managementbridge for services including transcoding voice communications and,optionally, pictures and maintaining communications with the supportcenter.

Another hardware configuration includes the concierge device withintegrated bridge. For instance, the concierge device may be a smartphone that runs a concierge application. If the consumer already owns anAndroid phone, an iPhone, an iPad or other portable device that includeswireless communication, a concierge application may be a low costimplementation of the concierge device. The service levels andassociation models described above still apply to concierge device withintegrated bridge.

In yet another configuration, the concierge device is built into theconsumer device, in the sense that the speaker and microphone are builtinto a major appliance, apropos to Internet surfing on a refrigeratordoor. In this configuration, the bridge could be built into the majorappliance (integrated bridge) or it could be a separate device coupledwirelessly or by powerline connection to the major appliance.

The home management bridge, again, optionally transcodes voicecommunications and, as a further option, pictures. The bridgeestablishes and maintains communications with the support center duringthe support call. It includes in the data stream with the support centeridentification information and, optionally, diagnostic information,consistent with the four service levels identified above or variationson those service levels.

DESCRIPTION OF FIGURES

FIG. 1 is a high level conceptual diagram that includes a conciergedevice, labeled Cloud Remote™, a home management bridge, a variety ofconsumer devices in a home and a service delivery platform connected bya network to the home. The service delivery platform can providefunctionality such as navigation, communication, personalizationsecurity, search and ad display, digital rights management, gaming andvirtual entertainment, data access and upload and place shift. Theservice delivery platform provides cloud-based management servers. Whilethere is a significant list of functions in this figure, many additionalfunctions could be supplied upon connecting a consumer device throughthe technology disclosed to services accessible via a network.

The concierge device may take on a variety of form factors as depictedin FIG. 2. In FIG. 2, from left to right, we see a wand, which includesone or more function buttons, a speaker and microphone. With voiceactivation, even the button might be eliminated, in favor of amicrophone and speaker. The device could turn on with motion sensinginstead of button activation, utilizing an electronic or MEMS sensor ofmotion or orientation such as an accelerometer, inclinometer, compass orgyroscope. In the middle is a simple concierge device with a thumb boardand navigation controls. To the right is a concierge device with a touchscreen. The concierge device can take on any of these form factors. Someof the scenarios described in this disclosure vary, depending on whetherthe form factor of the concierge device includes a display or not.

FIG. 3. depicts telephone and support service as functions that might beselected using a concierge device. A dedicated button on the conciergedevice might select a support call or navigation keys on the conciergedevice might be used with a single click to initiate a support call orsession.

FIGS. 4-8 show configurations of concierge device and bridge, eitherintegrated or standalone. FIGS. 5 and 6 depict scenarios using aconcierge device without a display. In these scenarios, AVR automatedtroubleshooting could be made available, in any of the modes describedabove. In FIG. 5, the concierge device can be associated directly withone of the consumer devices, such as a TV, light, refrigerator or washeror coupled to the consumer devices through the bridge. A computer, TV orother monitor can be used to confirm actions performed using theconcierge device or for configuration of the concierge device. Any ofthese monitors can be used for visually based automated troubleshooting, as described above. Alternatively, verbal commands andconfirmations could be exchanged between the concierge device and thehome management bridge. Upon request, the bridge can initiate a phonecall, using the concierge device as a handset. This support session willtypically be handled via VoIP through the Internet to a support server.Alternatively, cloud-based management servers coupled in communicationwith the concierge device could be used to initiate the phone call.

Optionally, the concierge device may communicate with the in-home bridgevia 511 through a built in or separate access point or through an inhome network. The concierge device can rely on the in-home bridge torelay its messages to other devices or to process its messages directly.In this scenario, a home management bridge handles communicationsbetween the concierge device and consumer devices. In someimplementations, the home management bridge also forwards signals fromhome control devices such as a wireless switch (SW) to controllabledevices, such as the TV, light, refrigerator or washer. The functions ofsuch home control devices may be replicated on the concierge device,especially when the device has a display, as in FIGS. 7-8.

In FIG. 6, the bridge is built into the television. Given thefunctionality included in televisions, the bridge described in thisdisclosure could readily be incorporated into a TV. Many TVs already arecapable of communication through the Internet to a server.Alternatively, the bridge could be built into the concierge device andthe TV used merely as a monitor to confirm actions being taken by theconcierge device.

FIG. 7 depicts a concierge device that includes a display, in use with ahome management bridge that is a separate and distinct physical device.Because this concierge device includes a display, it is unnecessary touse a personal computer or television to confirm actions being taken.Optionally, the concierge device may communicate with the in-home bridgevia 711 through a built in or separate access point and rely on thein-home bridge to relay its messages to other devices or process itsmessages directly.

FIG. 8 depicts a concierge device with an integrated bridge and display.In this figure, a touch screen device is depicted. The concierge devicemay be implemented in this configuration as an application running on asmart phone, tablet or similar device.

FIG. 4 depicts a concierge device built into a refrigerator or othermajor appliance. In this illustration, the bridge would be built intothe refrigerator as well as the concierge device. Alternatively, asdepicted in FIGS. 5-6, the bridge could be a standalone device or builtinto a TV or signal source for a TV, such as an A/V receiver, a DVD,cable or satellite box.

FIG. 9 is a flowchart of methods consistent with this disclosure. In theleft column are a series of actions. First, the concierge device isassociated with the consumer device. Several ways of handling thisassociation are described in the body and claims of this document. Asingle action using the concierge device initiates the supportactivities that follow. The single action could be a button press, asingle or double tap of a pad, a wave of the device or spoken word(s).Responsive to the single action, the concierge device causes a messageto be sent to the bridge. The bridge buffers consumer device informationfor use in a support call and then initiates the call, responsive to thesingle action. The bridge sends information to the support center insupport of the call. The information may be used to bypass automatedresponse trees and menus, to personalize the user experience. Theinformation optionally may be used to offer or initiate automatedtroubleshooting, as described above. The bridge also may send consumerdevice status information in aid of troubleshooting or, in some cases,even bridge a diagnostic session between a support server and theconsumer device being supported. In the right-hand column of FIG. 9, wesee alternative ways in which consumer electronic information can bebuffered to the bridge. It could be transferred via the conciergedevice. The bridge could communicate directly with the consumer device.Or, the consumer device information could be entered using any generaldata entry method, such as a keyboard or voice recognition. Optionally,the concierge device may communicate with the in-home bridge via 911through a built in or separate access point and rely on the in-homebridge to communicate relay its messages to other devices or process itsmessages directly.

FIG. 17 is a high level diagram of adding to support an automatedinteraction for the user. Building on FIG. 9, this figure illustratesusing the information sent to support to generate or select a relevantautomated interaction. For instance, the system sends the user adocument for display with troubleshooting hints and directions. Theinteraction may be optional, as some users may prefer guided troubleshooting to talking with a support person. Or, an automated protocol maybe a required preliminary that addresses frequently encountered issues.More alternative modes of automated or guided trouble shooting are givenabove. Optionally, the concierge device may communicate with the in-homebridge via 1711 through a built in or separate access point and rely onthe in-home bridge to communicate relay its messages to other devices orprocess its messages directly.

FIG. 18 represents a sample of information that may be supplied as partof an automated interaction. For instance, a variety of documentationsuch as manuals, data sheets, part lists and feature guides can beavailable. Pre-support checklists can be provided to cover steps such aschecking the power and rebooting the device. Support can be provided inresponse to general questions from a user. Alternatively, support can beprovided for diagnosed faults. These faults may be reported with statusor configuration data or diagnosed using status and configuration data.The data available from the consumer device can be combined with userresponses to focus the investigation. Self-help guidance can be providedfor those interested.

FIGS. 10A, 10B, 11A, 11B, 12A, 12B, 13A, 13B, 14A, 14B and 15 areexemplary screens for a concierge device that includes a display. InFIG. 10A users are presented with choices, including invoking aconcierge service to initiate a support call. Users are presented withchoices of watching live TV or IP TV, controlling consumer deviceswithin the home, initiating a VoIP call, playing games or invoking theconcierge service to initiate a support session.

FIG. 10B depicts selection of TV channels from live TV. Customizablebuttons can be provided and a user's favorite channels featured.

FIGS. 11A-B depict a video phone call. FIG. 11A is an example of acontacts list used to initiate calls. FIG. 11B is an example of a videophone call in progress.

FIG. 12A shows an example of user authentication using biometric data.In this example, use of a fingerprint for authentication is illustrated.Voice prints or facial recognition also can be implemented on aconcierge device, as can many forms of challenge and response.

FIG. 12B shows an example of an alert that a new home control device, aswitch is available for on-boarding. The home management bridge monitorsthe availability of new devices for on boarding. It can listencontinuously for new signals or can be placed in a listening, pairing orenrollment mode when a new device is to be added. The concierge devicereceive alerts from the bridge and alerts the user by visual, audible ortactile cues.

FIG. 13A is an example GUI that presents devices that a user can connectto the new switch.

FIG. 13B is an example GUI that used to select a device to control. Whena device is selected, another screen may appear that provides access tocontrols beyond a simple toggle.

FIG. 14A is an example of GUI from which a user can select a device andcause initiation of a support session. The devices illustrated as havingavailable concierge support include a TV, projector, home theater,radio, washer and oven. As easy as identifying devices for remotecontrol, the concierge panel allows a user to select a device and causeinitiation of a support telephone call. The call may be a video call oraudio-only. The system may automatically select the best communicationsavailable. A video call may include pictures from a front facing camera,directed towards the user, a back facing camera directed to the devicethat is being serviced or some combination of the two. The informationdisplayed from the support center may be directions, photographs of whatto look for, or the support person. As illustrated in other figures,interaction with a human support agent may be preceded by an automaticsupport protocol, optionally with diagnostic responses by the consumerdevice to queries received at the outset of a support session.

FIG. 14B is an example of a suggested fix presented to a user in asupport session prior to connection with a human agent. The sessionexample involves a television. Status information and, optionally,diagnostic responses to queries have been provided digitally to thesupport session. An automated support protocol has been invoked tosuggest possible resolutions of the user's problems. The problemsresolved may be suggested by the user, the support servers or both. Forinstance, in this example, the user might only have mentioned a videoresolution problem. Upon detecting a similar audio resolution problemthe suggested fix can be extended from the first problem that the userpresented to another, related problem. The user has the option of movingforward with human agent contact by pressing the “start call” button.

FIG. 15 depicts a video call with a human support agent.

FIG. 16 depicts types of information that may be useful to the supportcenter. In a home, devices 1 through N may be present. Controls 1through N may correspond to individual devices or persons within thehome. They may be linked through the bridge or directly to the Internet.The form factors for the concierge device may include any of the formfactors in FIG. 2, or built into a major appliance, or as an applicationrunning on a smart device such as a tablet or smart phone.Alternatively, a programmable face plate or remote could be mounted on aswitch box, cradled in a charger or surface mounted on the wall. May beportable, especially from the charger. A screen saver of this remotecould be a dimmer control. A home computer could also be used as aremote device, although the range of communications via Bluetooth orZigbee may be more difficult to accomplish between a stationary computerand a consumer device.

In the left-hand column, any number of persons may be located in thesame household as the consumer electronic and concierge devices.

More generally, the support service needs to handle multiple households.Consumer electronic information that may be of interest for servicepurposes includes the company that marketed the device, the manufacturerof the device, or the industry supported. For an individual device, thewarehouse that shipped the device and the device category may be ofinterest, as may be the retail store that sold it, and either thefabrication plant or contract manufacturer that produced the device. Ina full implementation, analytics hosted at a support center accessed viathe Internet or so-called cloud helps the support staff quickly diagnosethe problem being encountered. These analytics may take advantage ofinformation about the device manufacturing, but beyond the informationthat the consumer is able to supply. Given the reliable transmission ofinformation from the household through the bridge and network to thesupport center, it is anticipated that service will be significantlyimproved.

Problems and Solutions

The technology disclosed is useful to address a variety of problems. Forinstance, the problem of providing a user centric, multi-purposeinterface to diverse consumer devices (CA and CE devices) in a home canbe addressed using a handheld concierge device (a remote control) thatrelies redundantly or alternately on a home management bridge and/orcloud-based management servers, depending on their availability and theconfiguration selected. This concierge device sends input to and relieson the home management bridge, working in conjunction with thecloud-based management servers, to register devices in the home to bemanaged consumer devices. It accepts dynamically supplied menus verifiedto be up-to-date for control of the managed consumer devices. Thesemenus may be updated from time to time, either to fix bugs or as devicecapabilities are updated. It can act as a remote control for theconsumer devices, using the dynamically supplied menus. In someimplementations, it relies on the home management bridge or thecloud-based management servers to transcode multimedia to format used bythe remote control.

Another problem of providing efficiently connecting a user to technicalsupport can be solved by the technology disclosed using a low costconcierge device with minimal processing power, a microphone, a speaker,one to a few function buttons and a wireless transceiver; and by pairingthe concierge device with a consumer device, pressing one of thefunction buttons to initiate a call to a service center and utilizingresources in a bridge to 1) identify the consumer device and any currentstatus codes to the service center and 2) convert voice communicationsbetween the concierge device and the bridge to a VoIP and relay the VoIPcommunications to the service center.

One option is for the consumer device to communicate with the bridge,after setting up the bridge with household information and enrolling theconsumer device with the bridge. The bridge could be a computer or TVwith a wireless or powerline connection to particular consumer devicesand pairing protocols. A wireless connection could be direct or via amesh, per Zigbee, Zwave, Lutron ClearConnect™ or enOcean-likecommunication protocols. The wireless connection may be WiFi, 3G, BT,DECT, Zwave, Zigbee or virtually any RF that is low cost. Thedistinguishing protocol could involve promiscuous identification ofvisible consumer devices, graphic depiction from a cache or networkconnection of the consumer devices identified on a display, acceptingselection of a particular consumer device from a display, andinstructing a user to adjust one or more settings of the particularconsumer device to confirm pairing.

Consider the following example of on boarding a lamp to a home networkand using the concierge device to control the lamp. The in home gatewaydetects the new lamp and communicates with the Cloud Remote™ resultingin an alert such as, “detected a new connected device ‘Lamp’. To Acceptdevice please voice authenticate by saying ‘Confirm’” The user says“Confirm.” Simple menu screen is loaded onto the concierge devicefollowing the registration, which optionally may include authenticationby voice authentication. Optionally, a default or custom setup screenfor the lamp setup appears with a few parameters. User presses “DefaultsAccepted” on this screen. Of course, some devices may be so simple thatno user setup is required, beyond registration. Subsequent menus of“Devices” include the now-registered lamp. The user can turn the lamp onand off from this screen. The user also can on board a switch anddedicate that switch to controlling the lamp. Then, the user can useeither the switch or the concierge device to control the lamp.

The bridge alternatively could be a network appliance with a web serverthat is accessed from a computer or TV that provides a display. Thebridge has a wireless or powerline connection to particular consumerdevices and a pairing protocol that distinguishes consumer devices inthe household from consumer devices visible but outside the household.The distinguishing protocol could involve promiscuous identification ofvisible consumer devices, serving a graphic depiction from a cache ornetwork connection of the consumer devices identified on a display,accepting selection of a particular consumer device from a display, andinstructing a user to adjust one or more settings of the particularconsumer device to confirm pairing. The installation optionally includesreading a bar code on a device being coupled to the home managementbridge.

Another option is for the concierge device to communicate with both thepaired consumer device and a paired bridge, relaying information fromthe consumer device to the bridge.

The information relayed from the consumer device to the bridge could bepairing information that enables the consumer device and the bridge topair, based on the concierge device having paired with each, or at leastwith the bridge.

Alternatively, the bridge could be a computer with a Bluetooth adapterand a device pairing protocol for pairing with the concierge device. Thecomputer could set up the bridge with household information and enrollparticular consumer devices with the bridge, using the concierge deviceto manage pairing. Or, the bridge could be housed in a refrigerator orother major appliance. Some refrigerators now include a display and evena Windows-based operating system. Similarly, A/V receivers and DVDplayers include substantial resources that could be harnessed by abridge application and, optionally, a digital concierge application.

The concierge device could further include a camera that allows the userto show the service center a problem with the consumer device. Thebridge could cause instructional images to be relayed to a display onthe concierge device.

Sample Support Dialogs

Problems with a TV

-   -   1) Picture Resolution—In this dialogue, system configuration        information has been relayed from the consumer device to the        support center for use by the support person. The information        may be relayed when the support call is initiated, after it        begins, or even before the call, if it was sent by the consumer        device to the support center in advance, following the disclosed        setup. However the information is relayed to the support center,        it is available in the support call without the user needing to        verbalize or print and fax the configuration parameters. During        the session, the support staff remotely reconfigures settings on        the device. In practice, the dialog may require additional        permission steps, such as verbally granting permission or taking        an action using the concierge device or controls of the consumer        device to give permission to the support staff to adjust        settings. The user and support parts of the dialog are        illustrated below:

USER: I cannot see the corners of the picture on the screen. The pictureseems to be cropped.

SUPPORT: I see that even though your TV supports 1080P HD resolution,its resolution is set to 480i Video letter box mode. I have now adjustedthe resolution to be set to 1080P full screen mode when you operate yourset top box, Blu-Ray Player and gaming box which I see as the devicesconnected through your receiver to the screen. Can you see a full Testpattern on your screen now?

USER: Oh yes, thanks! The problem seems to have been resolved now.

-   -   2) Audio Fidelity—In this dialog, the support staff again uses        configuration information and remote configuration adjustment to        solve the user's problem.

USER: I cannot hear any sound from the rear speakers at all in mysurround sound system.

SUPPORT: I see that the audio on your receiver is set to Dolby 2.1 mode.I will now change that to Dolby 5.1. Can you confirm audio from the rearspeakers now?

USER: Oh yes, thanks! The problem seems to be resolved now.

-   -   3) Download issues—In this dialog, identification of the        household or user allows the support person to spot an issue        that has not generated a very informative error message.

USER: I don't seem to be able to download any movies. Can you pleasehelp me with that?

SUPPORT: Certainly. My Log shows that the Internet connection to yourhousehold has been up and running for the last 2 weeks without anyinterruptions. However, I am showing here that the primary credit cardon your file has expired. Would you like me to update that informationnow? I can link the card ending in digits 5202 as the valid credit card.Is that OK?

USER: Oh yes, thank you.

SUPPORT: You should be able to download any movies at this point.

USER: Thank you!

-   -   4) Gamebox connection and control—Information from multiple        devices is used for problem solving in this dialog. The bridge        has been configured to forward information to the support center        for more than just the Gamebox. This information may be        forwarded after configuration of the TV to work with the bridge,        as described above, or during the diagnostic session. Further        permission dialog may be required to cause status information to        be forwarded by more than just the Gamebox.

USER: Hi, I don't seem to be able to connect my Gamebox to my receiverand TV. Can you help me with that?

SUPPORT: Oh, certainly. I am showing here that your Gamebox reportscorrect functionality. However, your receiver is not set to select theproper video input when selecting the Gamebox. May I take correctiveaction for that?

USER: Yes, please.

SUPPORT: OK. The problem is now resolved. Also, your remote controlprogramming has been set the proper input settings for next time.

USER: Great, thank you.

-   -   5) Gifting or cross-selling—In combination with problem solving,        the support staff can either offer a gift or cross-sell a        product or service that a happy customer might be enticed to        buy:

If Picture problem, then Audio gift

If Audio problem, then Picture gift

Free movie/trailer download

-   -   6) Coupon: Support assistance redeeming a free music/movie        download coupon

SUPPORT: Can I help you with anything else?

USER: Oh yes . . . Can you tell me what this means? User holds up acoupon in front of the remote camera for which a video image is seen:coupon is recognized.

SUPPORT: Oh you're entitled to a free view of the new feature film“Help”. Would you like me to authorize that?

USER: Certainly.

SUPPORT: Anything else I can do?

USER: No thank you, appreciate the help

SUPPORT: You're welcome. And thank you for using CCC. Until next time,Goodbye!

USER: So long!

-   -   7) Market Research: survey followed by a free movie download        coupon

SUPPORT: Before we finish the call, I would like to offer you a freemovie download coupon in exchange for a few minutes of your time toconduct a brief survey on your most recent consumer device purchase.Would you like to accept the offer?

USER: Yes, of course.

SUPPORT: I am showing that the TV is your most recent purchase, is thatright?

USER: Yes, that is correct.

SUPPORT: What is the one feature that you're the most happy with?

USER: The fact that it has a very easy-to-use customer support feature!

SUPPORT: Thank you. What is the one feature you're the least happy with?

USER: The intrusive nature of firmware updates. I wish I did not have todeal with that!

SUPPORT: What is the one feature that is missing that you would love tohave on the next generation device?

USER: Automatic remote control configuration without me having to doanything or download an App.

SUPPORT: How can we improve our products or service for you?

USER: I wish I did not have to download a new app for every new device Ipurchase. And also, I am tired of trying to manage my applications . . .. This should all be a whole lot simpler given the degree ofconnectivity and intelligence in these devices.

SUPPORT: What is the next consumer device that you're planning topurchase?

USER: A Blu-ray player

SUPPORT: Thanks! We would like to show you a promotional video on ournew advanced Blu-Ray player that would be a perfect match for yourcurrent system. You will receive your coupon for free movie

Problems with the Washing Machine

-   -   1) Not washing properly

USER: My washing machine is working but the clothes do not smell cleananymore.

SUPPORT: I see that you've run 20 wash cycles without a clean cycle.Lingering bacteria from multiple washes builds up after a while. I wouldsuggest that you leave the washer door open after you're done with thewasher. You can also use clean cycle chemicals for maximum affect. I cansend you a sample . . . if you order online here's a coupon for 20% off.

-   -   2) Cross selling—The support staff uses supply level information        to offer refills, supplies and content in the following samples:

SUPPORT: Water filter on your refrigerator needs to be replaced. May Isend you a refill pack?

SUPPORT: AC filter needs to be replaced. May I send you a refill pack?

SUPPORT: There are free promotional downloads for the Blu-Ray playerthat you recently purchased. May I authorize the downloads for you?

-   -   3) Coupon for free sample detergent, similar to above.

SUPPORT: Can I help you with anything else?

USER: Oh yes . . . . Can you tell me what this means? User holds up acoupon in front of the remote camera for which a video image is seen:coupon is recognized

SUPPORT: Oh you're entitled to free samples of detergent and fabricsoftener. May I have them sent to you?

USER: Certainly.

SUPPORT: Anything else I can do?

USER: No thank you, appreciate the help

-   -   4) Market Research, as illustrated above.        High Level Description

The Concierge Connect Button

This button or other input sensor provides a convenient way for the userto request service for a device. When a support call is made using theconcierge device—without having to even look up the number—the serviceperson on the other side will be made aware of where the problem mostprobably lies. This can rely on Text, Voice or on Video+Voice. It is agame-changer when it comes to the convenience it provides for both theconsumer and the consumer device manufacturer. For the consumer, itmeans that no number needs to be looked up. The nature of the problemmay already be apparent in the main database, sorted out by cloudanalytics engines. Hours of wasted time can be prevented with thisscheme. For the service person, it means that he is no longer at themercy of the customer's description of the problem. The fact that thedevice is connected to the cloud and reports its status and any faultdata to the cloud automatically, without any consumer intervention,facilitates the diagnosis and resolution of the issue by the customerservice personnel, almost simultaneously as the call is made. For theconsumer device company, it means that unnecessary product returns canbe avoided. There is a reduced need to deeply educate the customerservice personnel on all fault possibilities for all devices. The cloudcomputing resources sort out the problem proactively and can evenprovide the service personnel with appropriate actions to be takenautomatically and proactively. The concierge device implementation neednot be limited to thin clients. It can be provisioned as an applicationrunning on a mobile phone, tablet, and PC or other connected device.

Device Management

Upon connecting the disclosed concierge device to the Internet, itbecomes possible to handle any other consumer device connected to thecloud. The concierge devices and methods of this disclosure affordseveral advantages. With this technology, any consumer device canconnect to the cloud securely. It can interact with the cloud securely.It can use a cloud-based UI and device management. The technologydisclosed caters to virtual apps: as soon as the consumer device isconnected to the cloud, it is fully configured; its pre-packaged UI isassociated with any concierge device, with its diagnostics/analyticspackage running in the cloud and ready to be leveraged. Menus, sometimescalled thin clients or virtual apps, for the consumer device can beimplemented on the concierge device dynamically or with a small cache.Such light weight virtual apps would not require user updating, thoughuser permission to update could optionally be required on a device lifeor per installation basis. Consumer devices coupled to a conciergedevice are ready to be personalized, based on associated users'profiles. The are ready to be leveraged in advertising generated in thecloud, whether for purchasing related accessories or viewing relevantmedia.

Enhanced Concierge Device Configuration

Manufacturers should be encouraged by this technology to join a singlecentralized service or, at least, to rely on a single concierge deviceand connection with cloud-based services. A single concierge deviceshould work with the consumer devices from at least a commonmanufacturer and also should work with several manufacturers' devices.The cloud could supply menus and remote control commands required forthe concierge device to work with any device from any manufacturer.

Advertising Options on a Concierge Device with a Display

The concierge device configurations that include displays can be used toshow advertising on the displays. U.S. application Ser. No. 11/350,980and 60/709,666 filed Feb. 8, 2006 and Aug. 19, 2005, respectively, bothentitled “Tethered Digital Butler Consumer Electronic Device and Method”depict devices that include displays. One of skill in the art willrecognize that chipsets have changed since those applications werefiled. At present, there are numerous suppliers of chipsets from which adesigner might choose, including Broadcom, Texas Instruments, Intel,Marvell, ST Microelectronics, Nvidia and IBM.

When the concierge device is coupled to a television or source for atelevision signal, the display on the concierge device can be asecondary or supplemental display for supplemental information. Ads maybe displayed as supplement information at opportune times. For instance,ads may be displayed while the user is fast forwarding rewinding orskipping through an ad or a series of advertisements. On the conciergedevice display, the user may see an ad logo, a program logo or sometext. Analytics may be applied to determine the best content for thesupplemental display. Alternatively, ads may be displayed while a useris viewing, fast forwarding or skipping through a program. The programlogo, an ad logo, or chapter location information may be displayed onthe supplemental display.

Ads that appear on the concierge device display may be synchronized withor independent of content being shown as the main program. For instance,when the main program reaches a sequence that involves productplacement, such as product placement for advertising purposes, acoordinated ad may appear on the supplemental display. That is, if Fordhas paid for a particular model of truck to appear in the program, theFord logo may appear on the supplemental display or a picture of theparticular truck may appear with a link for navigating to or rememberinginterest in that truck. Alternatively, ads for a competitor might appearon the supplemental display at the same time as the product placement.Similarly, ads may be directed to the supplemental display whileadvertising is being shown. On the supplemental display, the user mayhave a link or an option for remembering their interest in the productadvertised on the main display. Alternatively, ads for competitors mightappear on the supplemental display at the same time as the advertisingon the main display. A variety of competitive alternatives may beoffered in a single image on the supplemental display, coordinated witheither the product placement or advertising on the main display.

The ad content on the concierge device may be audio, video, pictures,graphics, or web content. It may appear during a program or duringadvertising within a program. The ad selected for display on thesupplemental screen may be program related, ad related, personalinformation related, user feedback related, user search related or userdevice related. Or, they may be served as banner ads, withoutconsideration of user identity or recent user activity.

A server connected by a network may supply analytics for ad selection orpersonalization. The server may recognize the format of display on theconcierge device or coordinated display and make sure that the ad isproperly sized. The server may control the timing of ad delivery andwhich channel, audio or video, and is used to display the advertising.For instance: advertisements may relate to searches performed by theuser via the remote. During a purchase cycle, ads may identify auxiliarydevice purchase suggestions. With an in-home device inventory, ads maydescribe available device upgrades based on existing devices at home.The ads may suggest media purchases related to existing devices

Advertising can be displayed on the concierge device's display while auser is watching a program on TV. For instance, ads may get the user'sattention while fast-forwarding through commercials. During thefast-forwarding, the concierge device may present an ad, an ad logo, orprogram logo, on the displayed of the remote. While fast-forwardingthrough a program, the program logo, or ad logo, can be displayed on theremote. Ads, can be directed to concierge device's display at anytime,independent of or dependent on—i.e., synced up—with the main program.Ads can play on the concierge device while a main program is running onmain screen. Such ads may be, or not, related to main program content.

Ads shown on the concierge device may be identical to or secondary toads a main ad is playing on main screen.

Virtually any content can be delivered by a concierge device with adisplay. The ad content may be audio, video, pictures or graphics oreven web content, in its many forms.

Ads may be presented on the concierge during a program showing on a mainscreen or as alternative content as an ad is on-screen.

Targeted or interactive ads can be selected. Many selection criteria anbe applied to ad display: ads may be program related, ad related,personal info related, user feedback related, user search related, userdevice related or general random.

Cloud-based servers can play a variety of roles in the systemsdisclosed. The cloud-based servers can provide analytics, ad selection,ad personalization, ad format, ad timing and/or an ad deliverymechanism. Virtually all ad-related functions can be served from thecloud.

Features of a concierge device can be combined in a variety of ways,depending on the configuration of the concierge device. In thisdisclosure, the concierge device is really a family of devicesconnecting to the cloud, as shown in the figures. A simple conciergedevice can include a microphone and speaker, without a display. On thissimple device, programming options include using local or remote voiceinteraction to do the programming based on prompts from the device. Or,the concierge device could be programmed using a PC.

An alternative concierge device configuration includes with a thumbboard or similar keyboard. In some implementations, this level of devicewill be similar to the simple concierge device, but with a keyboard forinput and display for output.

In some implementations, a touch screen is built into a simple conciergedevice. The touch screen provides for input/output.

The connection architecture of a concierge device depends on the extentto which bridge and routing capabilities are built in. With anintegrated router and bridge, a concierge device is direct to cloudready (DTCR). The concierge device can interact with server in the cloudusing a browser, such as a lightweight browser. Alternatively, with anintegrated bridge, the concierge device interacts direct to cloud viaBridge (DTCB). The bridge has the intelligence to set up communicationdestinations and sessions. In some implementations, a host is built intothe consumer device, such as a television or refrigerator. The conciergedevice communicates direct to cloud via Host (DTCH). It relies onintelligence built into the host.

Existing protocols can be adapted to the concierge devices capabilitiesand architectures described. Well-established protocols can be selectedto handle pairing, connection, authentication and communication.

When a device, whether a concierge device, a home control device, acontrollable device or a consumer device, is to be registered, cloudresource optionally are invoked. The concierge device can be registeredto a cloud-based server, just like any other device. An appliance getsregistered to cloud, either directly or by association with the remote.

Registration with a cloud based server can have utility in many ways. Acloud-based database can associate concierge devices with consumerdevices. It can record specifics of devices, including specifics ofconcierge devices with differing capabilities. When a concierge deviceinitiates a call, the database allows support personnel to know theuser, device and history of problems with the specific device and thedevice family. The database provides friendly script for the supportperson to follow.

One-touch support button can be realized, optionally accompanied byadvertising during the support session. By virtue of pre-existing remoteand device-aware database in the Cloud, a support person or automatedprotocol knows what to address and who is asking. This eliminatespreliminaries that sometimes dominate support sessions. The resultingsupport session may involve text exchanges, a voice or video connectionand/or feedback on the concierge device or an nearby monitor (TV or PC).The advertising on concierge device, as described above, can be targetedbased on key searches, programs being viewed, problematic appliancesand/or relevant appliance upgrades. Optionally, the concierge devicesupports voice commands which may be recognized locally or reported to aserver for recognition using, for instance, a VoIP connection.

A thin client model fits the concierge device architecture and hardware.A thin client can support the architectures identified above: direct tocloud (DTCR), device/cloud interaction through a lightweight browser;direct to cloud via Bridge (DTCB), in which the Bridge has theintelligence to set things up; or direct to cloud via Host (DTCH), inwhich the Host appliance has the intelligence. Going to the cloud isuseful for several reasons. The UI can be generated in the cloud or withminimal effort on the device. Ads can be targeted from the cloud tousers. Auto configuration routines and devices can be hosted andmaintained in the cloud.

A variety of additional features can be disclosed at this point. Someimplementations include tagging of devices so that concierge device canautomatically recognize the tagged devices based on proximity to device.Tags using RFID, near field communication or similar technology can besuitably employed.

Personalization can include user recognition and/or authentication basedon voice, fingerprints, RFID credentials, facial scans or event the waya user holds the concierge device, sensing the user's tremor pattern.Concierge devices can be initialized directly or in communication with aPC.

A direct-to-cloud API can support either remote procedure control or WebServices document exchange protocols to connect any device for cloudconcierge support.

Virtual Apps, user interfaces and device management can be consolidatedin the cloud. This consolidation would eliminate any need to downloadany apps. Consolidation in the cloud would assure that the conciergedevice was always up-to-date. The majority of UI requirements can bemanaged in the cloud, instead of on the local device. Learned userpreferences can be used to target ads.

Each device can be given a unique permanent serial number. This ID canbe programmed when the device is manufactured. The same database in thecloud can track a device from birth to death, from manufacturing toretail to home and on to when it is junked.

Services can be provided by a cloud-based platform. A premium humansupport service loosely analogous to GM's On-Star™ service for cars canbe provided for home appliances/gadgets/etc. Automatic configuration canbe fed to a connected device. Automatic UI generation can be hosted inthe cloud. Light weight client and simple interfaces eliminate any needto download any apps. Control can be implemented even on a conciergedevice without a display using voice commands and prompts. Appliancescan be shut down in cases of emergency. With the concierge device and/ora home management gateway acting in the background to monitor statuswithout user direction, fault notification can be delivered via email,text or other messaging medium, sometimes before the user notices aproblem.

Device configuration can be improved from defaults to take into accountdevices on the network that interacting with the device beingconfigured.

The concierge device has the advantage of inducing one or more uniformdata types, whether text, audio or video, which can be used to supportall devices of interest. Moreover, support for user authentication makesthe concierge device secure.

Example and Summary of Main Demo Screens

Home Screen

A variation on FIG. 10A can include icons for each of the following:

-   Cloud Connect Concierge Screen-   VoIP/Video Call Screen-   Credit Card Transaction/Security Screen-   Interactive Remote Display Screen

Touching an icon takes the user to a specific further screen.

Screen 1

FIG. 14A is the concierge support home screen. It showcases the essenceand ease-of-use with which support and services may be invoked. In someimplementations, clicking on a device highlights the device briefly andtransitions to Screen 2. In other implementations, screen 2 may bepreceded by an automated diagnostic protocol, as in the example of FIG.14B.

Screen 2

FIG. 15 is a webcam shot of a concierge support person at their desk.This leads to an interaction—with full audio and, optionally, withvideo—between the user and the service personnel.

There is a “Close Video Call” control which upon pressing transitions toScreen 3.

Screen 3

Following a support session, the display shows a summary for the problemjust resolved. The statement may be generic enough to be applicable toany device/appliance chosen on or it may track the problem and solutionreached during the support session. An ad may be positioned on thisscreen.

Screen 4

The VoIP/Video call screens, FIGS. 11A-B showcase a simple VoIP/Videoconinterface.

There may be a banner ad on top of the contacts list screen, FIG. 11A.

Upon touching a name, go to FIG. 11B. Alternatively, upon launching aconcierge support session, go to FIG. 15.

Screen 5

FIG. 15 is again a webcam shot of the concierge person at their desk.There is provision to turn video off for voice-only calls.

There may be a banner ad on top of this screen.

Screen 6

The Credit Card Transaction Screen has the picture of a credit card ontop. It is not shown in these figures.

Below it is a dotted box on the screen used for finger printauthentication purposes. FIG. 12A.

There may be a banner ad on top of the screen.

There can be a message saying the following: “Please touch the dottedbox for secure authentication, or simply say ‘I, John Smith, approve.’”

Upon touching the dotted box, a message pops us saying “TransactionApproved.”

Screen 7

FIG. 10A is a variation on an example interactive screen that displayssome form of the screen below with the following four icons: LiveTV,IPTV, Home Control, and VoIP (No Games/Chat/Contacts/Settings).

There may be a banner ad on top of the screen.

Some Particular Embodiments

The technology disclosed may be practiced as a method, device or articleof manufacture. One or more features of an implementation can becombined with the base implementation. Implementations that are notmutually exclusive are taught to be combinable. One or more features ofan implementation can be combined with other implementations. Thisdisclosure periodically reminds the user of these options. Omission fromsome implementations of recitations that repeat these options should notbe taken as limiting the combinations taught—these recitations arehereby incorporated forward by reference into each of the followingimplementations.

Single Action Support

Implementation #1

One group of methods, devices, systems and articles of manufacturerelates to using a concierge device to support consumer devices such asappliances and consumer electronics. In one implementation, a method isdescribed that includes initiating a support session for a consumerdevice using a concierge device. The method includes: associating theconcierge device with a selected consumer device; responsive to a singleaction performed by a user, the concierge device communicating with ahome gateway. Communicating with the home gateway includes: causing thehome gateway to buffer consumer device identification information forthe selected consumer device; and causing the home gateway to initiate asupport session for the consumer device and to forward automatically theconsumer device identification information during the support session,thereby allowing the support session either to bypass an automatedattendant or interactive voice recognition system or to initiate anautomated support protocol. Optionally, communicating with the homegateway further includes causing the home gateway to determine a supportcenter for a support session. This eliminates any need for the user tobe aware of the support center contact information.

This method and other implementations of the technology disclosed caneach optionally include one or more of the following features.

Associating can further include wirelessly pairing the concierge devicewith the selected consumer device and identifying the paired selectedconsumer device to the home gateway. Or, it can further includeselecting the consumer device from a menu of candidate consumer devicesknown to the home gateway. The menu can be an audio menu or be displayedvisually.

The home gateway can be a physical device separate and distinct from theconcierge device, to which the concierge device sends messages via anin-home network. Or, it can be part of the same physical device as theconcierge device. Alternatively, the home gateway can be a cloud-baseddevice to which the concierge device sends messages via a network thatextends outside the home.

The method can further include the concierge device receiving statusinformation from the selected consumer device and forwarding the statusinformation to the home gateway for use in the support session. Or,connecting with and receiving status information from the selectedconsumer device and using the status information in the support session.

The home gateway can feature relaying messages to and from the selectedconsumer device during the support session.

The method can further include displaying information received duringthe automated support protocol on a display of the concierge device. Theinformation can include supplemental information displayed as the useris participating in the support session. This supplemental informationcan include a targeted or interactive advertisement.

The information transmitted during the support session can enable a userof the concierge device to bypass interactive response systems and beginthe support session by talking to a human support agent, based on theforwarded consumer device identification information.

Other implementations may include a non-transitory computer readablestorage medium storing instructions executable by a processor to performa method as described above. Yet another implementation may include asystem including memory and one or more processors configured oroperable to execute instructions stored in the memory to perform amethod as described above.

Implementation #2

Another implementation of the technology disclosed is a concierge deviceincluding at least one wireless data communication adapter, one or moreinput sensors, one or more outputs, and memory; and at least oneprocessor configured to perform operations. Those operations areincluding: associating the concierge device with a selected consumerdevice; responsive to a single action performed by a user, the conciergedevice communicating with a home gateway. Communicating with the homegateway includes causing the home gateway to buffer consumer deviceidentification information for the selected consumer device; and causingthe home gateway to initiate a support session for the consumer deviceand to forward automatically the consumer device identificationinformation during the support session, thereby allowing the supportsession either to bypass an automated attendant or interactive voicerecognition system or to initiate an automated support protocol.Optionally, communicating with the home gateway further includes causingthe home gateway to determine a support center for a support session.

This device and other implementations of the technology disclosed caneach optionally include one or more of the following features.

Associating can further include wirelessly pairing the concierge devicewith the selected consumer device and identifying the paired selectedconsumer device to the home gateway. Or, it can include selecting theconsumer device from a menu of candidate consumer devices known to thehome gateway. The menu can be an audio menu or it can be displayedvisually.

The home gateway can be a physical device separate and distinct from theconcierge device and having its own processor, to which the conciergedevice is configured to send messages via an in-home network. Or, it canbe a part of the same physical device as the concierge device.Alternatively, the home gateway can be a cloud-based device to which theconcierge device sends messages via a network that extends outside thehome.

The processor of the concierge device can be further configured toperform operations that include displaying information received duringthe automated support protocol on a display of the concierge device. Theinformation can include supplemental information displayed as the useris participating in the support session. This supplemental informationcan include a targeted or interactive advertisement.

Generally, the processor(s) implementing this technology can beconfigured to perform operations reflecting any of forgoing methods andtheir aspects or features.

Implementation #3

Another implementation of the technology disclosed is a concierge devicethat includes: at least one wireless data communication adapter, one ormore input sensors, one or more outputs, and memory; and at least oneprocessor configured to perform operations. Those operations areincluding: linking the concierge device to data that uniquely identifiesa particular consumer device to be supported; and responsive to a singleuser action sensed by the input sensors, causing a home gateway toinitiate contact with a service center that supports the particularconsumer device, including automatically transmitting information fromwhich the service center can uniquely identify the particular consumerdevice and determine whether to engage an automated support protocolbased on the particular consumer device to be supported. The informationcan, for instance, be a serial number or locally unique assigned nameplus a concierge device, household or home gateway identifier.

This device and other implementations of the technology disclosed caneach optionally include one or more of the following features.

The device may be configured to perform operations including selectingthe particular consumer device to be supported from a plurality ofconsumer devices. It also may determine which of several support centersto initiate contact with to support the particular consumer device.

It may further include a display; and further be configured to performoperations including: automatically transmitting status informationregarding the particular consumer device to the service center; anddisplaying information received from the service center during theautomated support protocol.

The concierge device processor can further be configured to performoperations including: causing the particular consumer device to becoupled in data communication with the support center, thereby allowingdiagnostic communications between the particular consumer device and thesupport center. The communications between the particular consumerdevice and the support center can be relayed or direct, peer-to-peer.

In some implementations, the concierge device processor furtherconfigured to perform operations including conducting a support callwith the support processor regarding the particular consumer deviceusing the concierge device to capture the user's voice and to reproduceaudio for the user to hear.

Some implementations of the technology disclosed include a servicecenter that provides diagnostic support of a particular consumer device,the service center coupled in communication with the concierge device;wherein the service center includes at least one service processorconfigured to perform operations including: identifying the particularconsumer device from the automatically transmitted information; andengaging in an automated support protocol based on the particularconsumer device to be supported. alternatively, the service processorcan be configured to perform operations including: determining from theautomatically transmitted information to bypass automated support andconnect the user with a human; and engaging in at least audiocommunications with the concierge device.

Other implementations include a gateway device coupled in communicationwith the concierge device; wherein the gateway device includes at leastone gateway processor configured to perform operations including:relaying communications between the concierge device and the servicecenter; registering a plurality of consumer devices that are linkable tothe concierge device; and providing a menu from which to select of theparticular consumer device for the concierge device to support. The menuservices involve perceptible communications with the user using theinput sensors and outputs of the concierge device.

Implementation #4

Another implementation of the concierge device includes at least onewireless data communication adapter, one or more input sensors, one ormore outputs, and memory; and at least one processor configured toperform operations. The operations are including: linking the conciergedevice to data that uniquely identifies a particular consumer device tobe supported; responsive to a single user action directed to theconcierge device, initiating at least voice contact with a servicecenter that supports the particular consumer device, includingautomatically transmitting to the service center information from whichthe service center can uniquely identify the particular consumer device;and conducting a support call with the service center regarding theparticular consumer device using the concierge device to capture theuser's voice and to reproduce audio for the user to hear.

The concierge device processor can further be configured to performoperations including causing the particular consumer device to becoupled in communication with the support center during the supportcall.

The concierge device may further include a display on the conciergedevice that provides visual information to the user during the supportcall and/or a camera on the concierge device that provides visualinformation to the support center during the support call.

Implementation #5

The technology disclosed can be implemented in a consumer appliance withan integrated concierge device, based on any of the devices, aspects orfeatures described above. For instance, the concierge device can beintegrated into a refrigerator, washing machine, dryer, exerciseequipment, television, A/V center, stove, among other consumerappliances and electronic devices.

Generally, the technology can be implemented as a consumer appliancewith an integrated concierge device, including: a consumer device withappliance features, adapted to report status information; at least onenetwork adapter; a memory; at least one input sensor; at least one audioinput; an audio output; a processor coupled to the network adapter, thememory, the audio input, and the audio output; instructions running onthe processor. The instructions running on the processor may cause theintegrated device to communicate with a home management gateway; andcause the gateway to initiate a support call, responsive to a userinteracting with the input sensor, and to provide information regardingthe associated consumer device that allows the support call to bypass anautomated attendant or interactive voice recognition system and thatidentifies the associated consumer device to be supported. In someimplementations, the consumer device can have an integrated homemanagement gateway.

Other implementations may include a non-transitory computer readablestorage medium storing instructions executable by a processor to performa method as described above.

Yet another implementation may include a method including actionsimplemented by the processors of the device of system as describedabove.

Implementation #6

In another implementation, a method is described that includesinitiating a support session responsive to a single user action. Thisincludes linking a concierge device to data that uniquely identifies aparticular consumer device to be supported; and, responsive to a singleuser action sensed by one or more input sensors of the concierge device,initiating contact with a service center that supports the particularconsumer device, including automatically transmitting information fromwhich the service center can uniquely identify the particular consumerdevice and determine whether to engage an automated support protocolbased on the particular consumer device to be supported.

This method and other implementations of the technology disclosed caneach optionally include one or more of the following features.

The method may further include selecting the particular consumer deviceto be supported at a particular time from a plurality of consumerdevices and/or determining which of several support centers to initiatecontact with to support the particular consumer device.

Other implementations can include automatically transmitting statusinformation regarding the particular consumer device to the servicecenter; and displaying information received from the service centerduring the automated support protocol on a display of the conciergedevice.

Another optional feature includes causing the particular consumer deviceto be coupled in data communication with the support center, therebyallowing diagnostic communications between the particular consumerdevice and the support center. The communications may be relayed ordirect, peer-to-peer. Some implementations include conducting a supportcall with the support processor regarding the particular consumer deviceusing the concierge device to capture the user's voice and to reproduceaudio for the user to hear.

The method may further include actions of a service center, including:identifying the particular consumer device from the automaticallytransmitted information; and engaging in an automated support protocolbased on the particular consumer device to be supported. Alternatively,the service center can determine from the automatically transmittedinformation to bypass automated support and connect the user with ahuman; and to engage in at least audio communications with the conciergedevice.

The method may further include actions of a gateway device coupled incommunication with the concierge device including: relayingcommunications between the concierge device and the service center;registering a plurality of consumer devices that are linkable to theconcierge device; and providing a menu from which to select theparticular consumer device for the concierge device to support. It alsocan include menu services as describe above.

Implementation #7

In another implementation, a method is described that includes: linkinga concierge device to data that uniquely identifies a particularconsumer device to be supported; responsive to a single user actiondirected to the concierge device, initiating at least voice contact witha service center that supports the particular consumer device, includingautomatically transmitting to the service center information from whichthe service center can uniquely identify the particular consumer device;and conducting a support call with the service center regarding theparticular consumer device using the concierge device to capture theuser's voice and to reproduce audio for the user to hear.

The method can further include causing the particular consumer device tobe coupled in communication with the support center during the supportcall.

In some implementations, the method further includes a camera on theconcierge device displaying visual information from or sending itsending to the support center during the support call.

Linking can further include the concierge device interacting with anhome management gateway that maintains a copy of the data that uniquelyidentifies the particular consumer device. Or, it can include theconcierge device interacting with the home management gateway thatmaintains a copy of the data that uniquely identifies the particularconsumer device.

Automatically transmitting may include sending product model informationfor the particular consumer device and user identifiable information tothe support center.

In some implementations, the user could be directed in the support callto interact with an artificial intelligence that converses.

Implementations #8

Another implementation involves a method of initiating a service callfor a consumer device using a concierge device, including: pairing theconcierge device wirelessly with the consumer device; receiving from thepaired consumer device some consumer device identification information;and responsive to a single action performed by a user, the conciergedevice communicating with a home management gateway. Communicating withthe home management gateway includes forwarding the consumer deviceidentification information to the home management gateway; and causingthe gateway to initiate a support call for the consumer device and toforward the consumer device identification information during thesupport call, thereby allowing the support call to bypass an automatedattendant or interactive voice recognition system or to engage with anonline or alternative support protocol and identifying the associatedconsumer device to be supported.

Some implementations further include the concierge device receivingstatus information from the paired consumer device and forwarding thestatus information to the home management gateway for use during thesupport call.

Implementations #9-10

Another implementation involves a method of initiating a service callfor a consumer device using a concierge device, the method including:selecting the consumer device from a display of candidate consumerdevices known to a home management gateway. Following this initialselecting action, aspects and implementations of this method are as inthe preceding methods, aspects and implementations. Another alternativefor this initiating action includes: pairing the concierge devicewirelessly with the consumer device; receiving from the paired consumerdevice some consumer device identification information; and responsiveto a single action performed by a user, the concierge devicecommunicating with a home management gateway.

Another optional feature of this technology is the home managementgateway connecting with and receiving status information from theassociated consumer device and forwarding the status information to aservice center in advance of the service call.

Other implementations may include a non-transitory computer readablestorage medium storing instructions executable by a processor to performa method as described above. Yet another implementation may include asystem including memory and one or more processors configured oroperable to execute instructions stored in the memory to perform amethod as described above.

Implementation #11

One method involves initiating a support call for the consumer deviceusing the concierge device. This method begins with associating theconcierge device with the selected consumer device. Responsive to asingle action performed by user, the concierge device communicates withthe home management bridge. This communication causes the homemanagement bridge to buffer consumer electronic identificationinformation for the associated consumer device. It further causes thebridge to initiate a support call for the consumer device and to forwardthe consumer electronic information during the support call. Forwardingthis information, optionally, allows the support call to bypass anautomated attendant or interactive voice recognition system. Or, itcould be used to select a tailored interactive voice or visual protocolfor trouble shooting. It also could be used to initiate an artificiallyintelligent interaction using technology such as the Watson technologydemonstrated in 2011 by IBM. It further identifies the associatedconsumer device to be supported. This should greatly improve the usersupport experience.

We refer to associating the consumer device with the concierge device,because there are several ways in which this association may take place.The concierge device may be wirelessly paired with the selected consumerdevice. Identification of the consumer device may be programmed into theconcierge device memory, either at the factory or locally using a PC orsimilar device. The programming may be accomplished wirelessly using aUSB connection. Alternatively, the home management bridge can be used todisplay candidate consumer devices and select one for temporaryassociation with the concierge device. The concierge device can greatlysimplify on boarding and optionally provide authentication during an onboarding process, which would provide a degree of assurance to consumerswho are weary of home appliances connected to a cloud.

We refer to buffering consumer electronic identification information, asthere are several ways in which the bridge may obtain this information.First, the home management bridge may receive consumer electronic statusinformation associated with the consumer device from the conciergedevice, relayed to the bridge after pairing between the concierge deviceand the consumer device. Second, it may receive the information directlyby communicating with the consumer device identified by the conciergedevice, after pairing between the concierge device and the consumerdevice. Third, the concierge device may identify the consumer device tothe bridge and the bridge may obtain the information needed from theconsumer device directly. Fourth, the relevant information can be inputinto the bridge, using a conventional input, such as a keyboard, mouse,touch screen or voice recognition. Fifth, information can be obtainedfrom a cloud-based data source. For instance, the device may beregistered at its point of sale with a cloud-based management server.

Methods of initiating a support call can be more directly stated,beginning with the manner in which the concierge device is associatedwith the consumer device. One method variation of initiating a supportcall for the consumer device using the concierge device begins withpairing the concierge device wirelessly with the consumer device. Thosefamiliar with pairing Bluetooth headsets to wireless telephones orrecognize a wide variety of pairing options. The method proceeds withthe concierge device receiving from the paired consumer device someconsumer electronic identification information. Responsive to a singleaction performed by user, the concierge device communicates with thehome management bridge. It forwards consumer electronic information tothe home management bridge and causes the home management bridge toinitiate a support call. The bridge forwards the consumer electronicidentification information during the support call, thereby allowing thesupport call to bypass an automated attendant or interactive voicerecognition system. The bridge identifies the associated consumer deviceto be supported during the call.

Implementation #12

Another method of initiating a support call begins with selecting aconsumer device from the display of candidate consumer devices known tothe home management bridge. In this method, the home management bridgesupports multiple consumer devices. Responsive to a single actionperformed by user, the concierge device communicates with the homemanagement bridge. This causes the home management bridge to initiate asupport call for the selected consumer device and to forward consumerelectronic identification information to the support center. Theconsumer electronic identification information can be obtained either bycommunications between the bridge and the selected consumer device or byentry of information through the bridge. The information may be providedin advance of the phone call, so that only a serial number or otherglobally unique identifier needs to be provided.

Implementation #13

A variety of devices and systems also practice the disclosed technology.One is the concierge device that initiates support calls for associatedconsumer devices. The support calls are initiated by the home managementbridge. The concierge device includes at least one wireless transceiveradapted to be coupled in communication with at least one home managementbridge. The concierge device further includes a memory, at least onebutton or other single action control, speaker and microphone, and aprocessor coupled to the transceiver, the memory, the button, thespeaker and microphone. Examples of single action controls include abutton, toggle or other physical switch, touch areas on a touch screen,a motion sensor or a voice activated control. Instructions are providedrunning on the processor that wirelessly couples the concierge device tothe home management bridge. One skilled in the art will recognize thatprocessor is a broad term, which may include both the CPU and thewireless communications components. The instructions further forwardconcierge device information to the home management bridge and cause thebridge to initiate a support call, responsive to a single user actionsuch as pressing a button. Instructions further cause a bridge toprovide information regarding the associated consumer device that allowsthe support call to bypass an automated attendant or interactive voicerecognition system and that identifies the associated consumer device tobe supported. The identification may be direct or indirect, specific orcategorical. Preferably, it would include the serial number of theconsumer device, to facilitate verification of support eligibility andis data for subsequent analytics at the support center.

As one option, the wireless transceiver may be adapted to be coupled incommunication with at least one associated consumer device. Instructionsrunning on the processor of the concierge device may further wirelesslycouple the concierge device to the associated consumer device and obtainfrom the associate consumer device some consumer electronicidentification information. Then, the instructions cause the consumerelectronic information to be forwarded to the home management bridge,which in turn is relied upon to forward the consumer electronicidentification information during the support call.

A further aspect of this option is that the instructions running on theprocessor may further relay diagnostic status information from theconsumer device to the home management bridge for use in the supportcall.

As another option, the concierge device may include at least onewireless transceiver adapted to be coupled in communication with atleast one consumer device. Instructions running on its processor furtherwirelessly couple the concierge device to the consumer device, obtainsome consumer electronic identification information from the device andforward this information to the bridge. The instructions further causeand rely on the bridge to pair with the consumer device using theconsumer electronic identification information. This makes informationavailable to the home management bridge for use in the support call.

As illustrated in FIG. 4, the concierge device may be built into arefrigerator or other major appliance. In this illustration, the bridgewould be built into the refrigerator as well as the concierge device.Alternatively, as depicted in FIGS. 5-6, the bridge could be astandalone device or built into a TV or signal source for a TV, such asa A/V receiver, a DVD, cable or satellite box.

Yet another option is for the concierge device to have consumerelectronic identification information loaded into its memory.Information from the memory of the concierge device may be forwarded tothe home management bridge for use during the support call.

In connection with any of the methods and devices described above, theconcierge device further may include a camera coupled to the processor.Instructions running on the processor further will wirelessly couple thecamera with the home management bridge and cause the home managementbridge to forward images from the camera during the support call. Theimages may be still or moving images.

In further connection with the methods and devices described above, theconcierge device may include a display coupled to the processor.Instructions running on the processor may cause the concierge device toreceive and display images received from the home management bridgeduring the support call. Combining the camera and display featuresdescribed in these successive paragraphs essentially provides a two-wayvideoconference.

The concierge device as described above may be combined into a systemthat includes a home management bridge. Such a home management bridgeincludes at least one wireless transceiver adapted to be coupled incommunication with at least one concierge device, a method, networkinterface and a processor coupled to each of the above. Instructionsrunning on the processor wirelessly couple the home management bridge tothe concierge device and receive concierge device information from theconcierge device. The instructions are further adapted to initiate asupport call, responsive to a request from the concierge device, and toprovide information regarding the associated consumer device to asupport system that allows the support call to bypass an automatedattendant or interactive voice recognition system. The informationfurther identifies the associated consumer device to be supported.

In some sense, the concierge device described may act as the eyes, earsand voice of the bridge or cloud based service for interacting withusers across a wide variety of consumer device, making the connected,intelligent home a manageable reality for consumers.

On Boarding Claims

Another group of methods, devices, systems and articles of manufacturerelates to using a concierge device to on-board home control devices andcontrollable devices to a home management gateway.

Implementation #14

One implementation describes a method of on boarding to an in-homemanagement bridge both a control device and a controlled device. Thismethod is including: receiving a first enrollment signal from the firsthome control device that uniquely identifies the first home controldevice, automatically determining control capabilities of the first homecontrol device and enrolling the first home control device with themanagement bridge; receiving a second enrollment signal from the secondcontrollable device that uniquely identifies the second controllabledevice, automatically determining control signals recognized by thesecond controllable device and enrolling the second controllable devicewith the management bridge; and receiving a dedication signal responsiveto a user interface and automatically configuring a combination of themanagement bridge, the first home control device and the secondcontrollable device so that at least some control signals generated bythe first home control device are dedicated to controlling the secondcontrollable device.

This method and other implementations of the technology disclosed caneach optionally include one or more of the following features.

The method can further include automatically replicating the controlcapabilities of the first home control device on a concierge devicecoupled in communication with the management bridge. This can furtherinvolve displaying the user interface used to generate the dedicationsignal on a display of the concierge device.

Some implementations include displaying supplemental information on adisplay of the concierge device. This supplemental information can be atargeted or interactive advertisement.

The method can involve the management bridge requesting of and receivingfrom a cloud-based server at least some configuration data used whenautomatically configuring the combination.

The management bridge may take on alternative configurations. In oneimplementation, the management bridge is a physical device separate anddistinct from the concierge device; and the concierge device sendsmessages to and receives messages from the management bridge via anin-home network. In another implementation, the management bridge ispart of the same physical device as the concierge device.

Another implementation includes scanning packaging or a tag accompanyingeither the first home control device or the second controllable deviceto generate the first or second enrollment signal, respectively.Scanning may be implemented in a variety of ways, including opticallyscanning a pattern on a label or wirelessly scanning an RFID tag.

The dedicating action can further include receiving one or more userselections responsive to the user interface that select the first homecontrol device from a plurality of home control devices and that selectthe second controllable device from a plurality of controllable devices.

Alternative messaging patterns can be implemented. The dedicating actioncan further include configuring one or more of the first home controldevice and the second controllable device for peer-to-peer controlinteraction. Or, it can include configuring the management bridge toreceive the first control signals from the first home control device andsend the second control signals to the second controllable device,responsive to the first control signals.

Inventorying devices to enroll can precede receiving enrollment signals.Some implementations further include: establishing an inventory ofuniquely identified home control devices and controllable devices withthe management bridge, the inventory including at least the first homecontrol device and the second controllable device; matching informationin the first enrollment signal to the inventory during the enrolling ofthe first home control device with the management bridge; and matchinginformation in the second enrollment signal to the inventory during theenrolling of the second controllable device with the management bridge.The inventorying can further include scanning packaging or a tagaccompanying either the first home control device or the secondcontrollable device to inventory the first home control device or thesecond controllable device, respectively, and the options identifiedabove for scanning.

Some implementations further include authenticating a user prior toreceiving the identification of the unregistered unit and initiating anenrollment mode in which enrolling and dedicating are permitted.Authenticating can require the user to physically interact with aconcierge device that previously was enrolled with the management bridgeor that implements the enrollment with the management bridge.Alternatively or in addition, authenticating can require receivingbiometric information from the user.

Other implementations may include a non-transitory computer readablestorage medium storing instructions executable by a processor to performa method as described above. Yet another implementation may include asystem including memory and one or more processors configured oroperable to execute instructions stored in the memory to perform amethod as described above.

Implementation #15

Another implementation of the technology disclosed is a system with ahome management bridge including at least one network communicationadapter, and memory; and at least one processor configured to performoperations. The operations are including: receiving a first enrollmentsignal from the first home control device that uniquely identifies thefirst home control device, automatically determining controlcapabilities of the first home control device and enrolling the firsthome control device with the management bridge; receiving a secondenrollment signal from the second controllable device that uniquelyidentifies the second controllable device, automatically determiningcontrol signals recognized by the second controllable device andenrolling the second controllable device with the management bridge; andreceiving a dedication signal responsive to a user interface andautomatically configuring a combination of the management bridge, thefirst home control device and the second controllable device so that atleast some control signals generated by the first home control deviceare dedicated to controlling the second controllable device.

This system and device and other implementations of the technologydisclosed can each optionally include one or more of the featuresdescribed above in the method context.

Some implementations involve a system including the home managementbridge and further including a concierge device that comprises at leastone input sensor and at least one output, in which the processor isfurther configured to perform operations including automaticallyreplicating the control capabilities of the first home control device ona concierge device coupled in communication with the management bridge.

In some implementations of this system, the concierge device furtherincludes at least one processor configured to perform operationsincluding displaying the user interface used to generate the dedicationsignal on a display of the concierge device.

The concierge device in this system can further include at least oneprocessor configured to perform operations including displayingsupplemental information on a display of the concierge device. Thesupplemental information can be a targeted or interactive advertisement.

Systems of this sort can further include a scanning device configured toscan packaging or a tag accompanying either the first home controldevice or the second controllable device to generate the first or secondenrollment signal, respectively.

In some implementations, the dedicating action further includesreceiving one or more user selections responsive to the user interfacethat select the first home control device from a plurality of homecontrol devices and that select the second controllable device from aplurality of controllable devices.

In some implementations, the management device processor is furtherconfigured to perform operations including authenticating a user priorto receiving the identification of the unregistered unit and initiatingan enrollment mode in which enrolling and dedicating are permitted.Authenticating can require the user to physically interact with theconcierge device, which previously was enrolled with the managementbridge or which implements the enrollment with the management bridge.Alternatively or in addition, authenticating can require receivingbiometric information from the user.

Other implementations may include a non-transitory computer readablestorage medium storing instructions executable by a processor to performa method as described above. Yet another implementation may include amethod including actions implemented by the processors of the device ofsystem as described above.

Implementation #16

Another method implementation includes: receiving an identification ofan unregistered in-home unit that uniquely identifies the unit, whereinthe unit is an in-home appliance control device or controllable deviceand control devices are complementary to controllable devices;registering the unit using the identification received; and receivingintended connection information and configuring a connection between theregistered unit to a previously registered complementary unit toestablish a unit pair, wherein the connection causes a controllable unitof the unit pair to be controlled responsive to control signals sent bya control unit of the unit pair.

Configuring the connection can further include configuring a bridgedevice to receive signals from the control unit and relay them to thecontrollable unit accompanied by an identifier accepted by thecontrollable unit. In some implementations, the bridge device adds theidentifier accepted by the controllable unit to a control signal.

The bridge device can be physical distinct from and not incorporated ineither the control unit or the controllable unit. It can be incorporatedinto a concierge device.

Configuring the connection can further include configuring thecontrollable unit to accept signals sent by the control unit that areaccompanied by a control unit identification. When the unit registeredis the control unit, registering the control unit can further includeidentifying control signals generated by the control unit. When the unitregistered is the controllable unit, registering the controllable unitcan further include identifying control signals accepted by thecontrollable unit.

In some implementations, the method further includes configuring aprogrammable home control device with a display: to allow user selectionof the control device from the display; to allow user selection ofcontrols provided by the control unit; and to send control signals fromthe home control device that cause the controllable unit to beresponsive as if the signals had been sent by the control unit.

The method can further include authenticating a user prior to receivingthe identification of the unregistered unit and initiating aregistration mode in which registering and connecting are permitted. Asdescribed above, authenticating can require interaction with theconcierge device and/or biometric information from the user.

Some implementations further include: registering a plurality ofunregistered control units and a plurality of unregistered controllableunits; and configuring connections among the control and controllableunits after registering the plurality of units.

Identifications of the plurality of unregistered units can be obtainedby scanning packaging or tags accompanying with the unregistered units,applying any conventional scanning technique such as those mentionedabove.

Other implementations may include a non-transitory computer readablestorage medium storing instructions executable by a processor to performa method as described above. Yet another implementation may include asystem including memory and one or more processors configured oroperable to execute instructions stored in the memory to perform amethod as described above.

Implementation #17

Some further device implementations include at least one networkcommunication adapter, one or more input sensors, one or more outputs,and memory; and at least one processor configured to perform operations.The operations are including: receiving an identification of anunregistered in-home unit that uniquely identifies the unit, wherein theunit is an in-home appliance control device or controllable device andcontrol devices are complementary to controllable devices; registeringthe unit using the identification received; and receiving intendedconnection information and configuring a connection between theregistered unit to a previously registered complementary unit toestablish a unit pair, wherein the connection causes a controllable unitof the unit pair to be controlled responsive to control signals sent bya control unit of the unit pair.

This device and other implementations of the technology disclosed caneach optionally include one or more of the device features describedabove.

Implementation #18

Several implementations involve inventorying units before enrollingthem. One of these implementations describes a method of on boarding andinterconnecting home control devices with controllable devices thatincludes: establishing an inventory of uniquely identified home controldevices and controllable devices with an in-home management bridge, theinventory including at least a first home control device and a secondcontrollable device; receiving a first enrollment signal from the firsthome control device that uniquely identifies the first home controldevice, matching information in the first enrollment signal to theinventory, and enrolling the first home control device with themanagement bridge; receiving a second enrollment signal from the secondcontrollable device that uniquely identifies the second controllabledevice, matching information in the second enrollment signal to theinventory, and enrolling the second controllable device with themanagement bridge; receiving a dedication signal responsive to a userinterface and automatically configuring a combination of the managementbridge, the first home control device and the second controllable deviceso that control signals generated by the first home control device arededicated to controlling the second controllable device; and replicatingcontrol capabilities of the first home control device on a conciergedevice coupled in communication with the management bridge.

The dedicating action can further include configuring one or more of thefirst home control device and the second controllable device forpeer-to-peer control interaction.

Some implementations further include automatically determining firstcontrol signals generated by the first home control device and secondcontrol signals that control the second controllable device; wherein thededicating action further includes configuring the management bridge toreceive the first control signals from the first home control device andsend the second control signals to the second controllable device,responsive to the first control signals.

Some implementations further include providing an interface that enablesa user to select dedication of control signals generated by the firsthome control device to control of the second controllable device.

An alternative implementation describes a method of on boarding anin-home controllable device to a management bridge and automaticallyconfiguring a concierge device to control the controllable device,including: establishing an inventory of uniquely identified controllabledevices with an in-home management bridge, the inventory including atleast a first in-home controllable device; receiving a first enrollmentsignal from the first in-home controllable device that uniquelyidentifies the first controllable device, matching information in thefirst enrollment signal to the inventory, and enrolling the firstcontrollable device with the management bridge; and automaticallydetermining first control signals used to control the first controllabledevice and configuring a concierge device coupled in communication withthe management bridge with one or more menu items that control the firstcontrollable device using the first control signals.

Another alternative implementation describes a method of on boarding anin-home control device to a management bridge and automaticallyconfiguring a concierge device to replicate control capabilities of thecontrollable device, including: establishing an inventory of uniquelyidentified control devices with an in-home management bridge, theinventory including at least a first in-home control device; receiving afirst enrollment signal from the first in-home control device thatuniquely identifies the first control device, matching information inthe first enrollment signal to the inventory, and enrolling the firstcontrol device with the management bridge; and automatically determiningfirst control signals generated by the first control device andconfiguring a concierge device coupled in communication with themanagement bridge with one or more menu items that replicate controlcapabilities of the first control device.

Other implementations may include a non-transitory computer readablestorage medium storing instructions executable by a processor to performa method as described above. Yet another implementation may include asystem including memory and one or more processors configured oroperable to execute instructions stored in the memory to perform amethod as described above.

Implementation #19

Another implementation describes a method of on boarding to an in-homemanagement bridge both a control device and a controlled device. Thismethod includes: establishing an inventory of uniquely identified homecontrol devices and controllable devices with an in-home managementbridge, the inventory including at least a first home control device anda second controllable device; receiving a first enrollment signal fromthe first home control device that uniquely identifies the first homecontrol device, matching information in the first enrollment signal tothe inventory and enrolling the first home control device with themanagement bridge; receiving a second enrollment signal from the secondcontrollable device that uniquely identifies the second controllabledevice, matching information in the second enrollment signal to theinventory, and enrolling the second controllable device with themanagement bridge; receiving a dedication signal responsive to a userinterface and automatically configuring a combination of the managementbridge, the first home control device and the second controllable deviceso that at least some control signals generated by the first homecontrol device are dedicated to controlling the second controllabledevice.

As above, this method and other implementations of the technologydisclosed can each optionally include one or more of the followingfeatures.

Some implementations further include replicating control capabilities ofthe first home control device on a concierge device coupled incommunication with the management bridge.

Dedicating can further include configuring one or more of the first homecontrol device and the second controllable device for peer-to-peercontrol interaction. Or, it can involve automatically determining firstcontrol signals generated by the first home control device and secondcontrol signals that control the second controllable device, andconfiguring the management bridge to receive the first control signalsfrom the first home control device and send the second control signalsto the second controllable device, responsive to the first controlsignals.

Implementations also can include providing an interface that enables auser to select dedication of control signals generated by the first homecontrol device to control of the second controllable device.

Another implementation describes a method of on boarding an in-homecontrollable device to a management bridge and automatically configuringa concierge device to control the controllable device, including:establishing an inventory of uniquely identified controllable deviceswith an in-home management bridge, the inventory including at least afirst in-home controllable device; receiving a first enrollment signalfrom the first in-home controllable device that uniquely identifies thefirst controllable device; matching information in the first enrollmentsignal to the inventory; and enrolling the first controllable devicewith the management bridge.

This automatic configuring method can optionally include one or more ofthe features described above. It also can involve automaticallydetermining first control signals used to control the first controllabledevice and configuring the concierge device coupled in communicationwith the management bridge with one or more menu items that control thefirst controllable device using the first control signals.

Some implementations involve on boarding either an in-home controldevice or a controllable device to a management bridge. When just thein-home control device is enrolled, the method can include automaticallyconfiguring a concierge device to replicate controls of the controldevice; establishing an inventory of uniquely identified control deviceswith an in-home management bridge, the inventory including at least afirst in-home control device; receiving a first enrollment signal fromthe first in-home control device that uniquely identifies the firstcontrol device; matching information in the first enrollment signal tothe inventory; and enrolling the first control device with themanagement bridge.

When just the in-home controllable device is enrolled to a managementbridge, the method can include automatically configuring a conciergedevice to control the controllable device; receiving a first enrollmentsignal from the first in-home controllable device that uniquelyidentifies the first controllable device; automatically determiningfirst control signals used to control the first controllable device; andenrolling the first controllable device with an in-home managementbridge.

Some implementations further include automatically configuring aconcierge device coupled in communication with the management bridgewith one or more menu items that control the first controllable device,implementing at least some of the first control signals.

Another method of on boarding an in-home control device to a managementbridge and automatically configuring a concierge device to replicatecontrol capabilities of the control device, includes: receiving a firstenrollment signal from the first in-home control device that uniquelyidentifies the first control device, automatically determining firstcontrol signals used to control the first controllable device, andenrolling the first controllable device with an in-home managementbridge; automatically determining first control signals generated by thefirst control device; and configuring a concierge device coupled incommunication with the management bridge with one or more menu itemsthat replicate control capabilities of the first control device.

Other implementations may include a non-transitory computer readablestorage medium storing instructions executable by a processor to performa method as described above. Yet another implementation may include asystem including memory and one or more processors configured oroperable to execute instructions stored in the memory to perform amethod as described above.

Virtual Apps and Advertising

Implementation #20

Some implementations describe a concierge device that redundantlyconnects to an associated consumer device to an in-home managementbridge or to cloud-based management server. The concierge deviceincludes: at least one wireless transceiver and network protocol stackadapted to be coupled in communication with at least a home managementbridge and cloud-based management server; a memory; at least one userinput sensor (which optionally could be the microphone identifiedbelow); a display; a speaker and microphone; a processor coupled to thetransceiver, the memory, the input sensor, and the display; instructionsrunning on the processor. The instructions, when executed, cause theconcierge device to: wirelessly couple the concierge device to thein-home management bridge; wirelessly couple the concierge device to thecloud-based management server; sense availability or unavailability ofthe in-home management bridge and the cloud-based management server;preferentially rely on one of the in-home management bridge or thecloud-based management server and alternatively rely on the other one;register the consumer device through the in-home management bridge oralternatively through the cloud-based management server; responsive tothe registration of the consumer device, receive one or more controlmenus for the consumer device and control strings to send directly orindirectly to the consumer device to implement selections from thecontrol menus; and responsive to actions of a user, send directly orindirectly to the consumer device the control strings.

This device and other implementations of the technology disclosed caneach optionally include one or more of the following features.

The concierge device can be coupled in communication with at least oneconsumer device through the in-home management bridge. Thiscommunication can be direct, through the in-home management bridge orthrough a wireless access point.

The user input sensor can be a button, a motion sensor, an orientationsensor, or a microphone.

In some implementations, the concierge device accepts control menus forcontrol of appliances, including at least a refrigerator, dishwasher andwashing machine.

The concierge device describe above may further include instructionsrunning on the processor that: forward concierge device identificationinformation to the in-home consumer management bridge or alternativelyto the cloud-based consumer management services, depending onavailability; and cause the in-home consumer management bridge or thecloud-based consumer management services to initiate a support call,responsive to a user pressing the button, and to provide informationregarding the associated consumer device that allows the support call tobypass an automated attendant or to invoke an interactive voicerecognition system using information specific to the associated consumerdevice.

In some implementations, the concierge device includes instructions thatcause the concierge device to store information particular to aplurality of consumer devices as they are registered and to make thatinformation available for control or support of the consumer deviceseither directly or via the bridge, responsive to selection of aparticular associated consumer device.

Instructions running on the processor can cause the processor of theconcierge device to receive alert messages regarding completion of atask by an in-home consumer device. Instructions running on theprocessor also can cause the processor of the concierge device toreceive alert messages regarding interruption of a task being performedby an in-home consumer device. The concierge device can be configured toemit an audible alert or display a message in response to the alertmessages.

Instructions running on the processor further can cause the processor ofthe concierge device to check for updates to the control menus withoutneed for user intervention.

Implementation #21

Another implementation describes a method including: receiving anidentification of an unregistered consumer device from which at least amake and model of the consumer device can be determined; registering theconsumer device using the identification received; and automaticallyconfiguring a home control device with menus and control codes thatcontrol the consumer device and with support information, wherein thehome control device includes one or more input sensors and a display.

This implementation optionally includes, responsive to a single useraction sensed by the input sensors, the home control device initiatingcontact with a service center that supports the consumer device,including automatically transmitting information from which the servicecenter can identify the consumer device and determine whether to engagean automated support protocol based on the consumer device to besupported.

This method and other implementations of the technology disclosed caneach optionally include one or more of the following features.

Other implementations include: automatically transmitting statusinformation regarding the consumer device to the service center; anddisplaying information received from the service center during theautomated support protocol.

In some implementations, the wherein the home control device is coupledin communication with the service center by an in-home support gateway;and the method further includes the in-home support gateway relayingcommunications between home control device and the service center.

In other implementations, the home control device is coupled incommunication with a cloud-based support gateway via a network thatextends beyond the home and the method further includes the cloud-basedsupport gateway implementing the registering of the consumer device andthe automatically configuring of the home control device.

Some implementations further include causing the consumer device to becoupled in data communication with the support center, thereby allowingdiagnostic communications between the consumer device and the supportcenter.

A support call can be conducted using the home control device to capturethe user's voice and to reproduce audio for the user to hear. It can useat least one video camera on the home control device to provide visualinformation to the support center during the support call.

Further implementations include the home control device receiving a datastream from the consumer device and displaying information from the datastream on the home control device in coordination with interactionsbetween the home control device and the consumer device.

When the consumer device is a television or video image source coupledto a monitor, the information displayed on the home control devicedisplay can be coordinated with images displayed on the television ormonitor.

The information displayed on the home control device can identify acompany or product and can be coordinated with product placement in anadvertisement or program scene. Or it can be coordinated with subjectmatter of an advertisement or program scene.

Information can be displayed on the home control device as a usernavigates through content by fast forwarding, rewinding or skipping.

Implementation #22

Some implementations describe a system including: at least one wirelessdata communication adapter, one or more input sensors, one or moreoutputs, and memory; and at least one processor configured to performoperations. The operations are including: receiving an identification ofan unregistered consumer device from which at least a make and model ofthe consumer device can be determined; registering the consumer deviceusing the identification received; and automatically configuring a homecontrol device with menus and control codes that control the consumerdevice and with support information, wherein the home control deviceincludes one or more input sensors and a display. In this system,responsive to a single user action sensed by the input sensors, the homecontrol device initiating contact with a service center that supportsthe consumer device, including automatically transmitting informationfrom which the service center can identify the consumer device anddetermine whether to engage an automated support protocol based on theconsumer device to be supported.

This system and other implementations of the technology disclosed caneach optionally include one or more of the following features.

In some implementations, the system further includes the processorconfigured to perform operations including: automatically transmittingstatus information regarding the consumer device to the service center;and displaying information received from the service center during theautomated support protocol.

Some system implementations further include an in-home support gatewaythat comprises at least one gateway processor configured to performoperations including relaying communications between home control deviceand the service center. In these implementations, the home controldevice can be coupled in communication with the service center bythrough the in-home support gateway. The gateway processor can befurther configured to perform operations that implement the registeringof the consumer device and the automatically configuring of the homecontrol device.

Other system implementations further include a cloud-based supportgateway that comprises at least one gateway processor configured toperform operations that implement the registering of the consumer deviceand the automatically configuring of the home control device. In theseimplementations, the home control device is coupled in communicationwith the cloud-based support gateway via a network that extends beyondthe home.

The processor used can be further configured to perform operationsincluding causing the consumer device to be coupled in datacommunication with the support center, thereby allowing diagnosticcommunications between the consumer device and the support center.

The processor on the home control device can be further configured toperform operations including conducting a support call with the supportprocessor regarding the consumer device using the home control device tocapture the user's voice and to reproduce audio for the user to hear.The processor on the home control device can be further configured toperform operations including using at least one video camera on the homecontrol device to provide visual information to the support centerduring the support call.

Any of these implementations can further include the processor on thehome control device is further configured to perform operationsincluding receiving a data stream from the consumer device anddisplaying information from the data stream on the home control devicein coordination with interactions between the home control device andthe consumer device. When the consumer device is a television or videoimage source coupled to a monitor, the information displayed on the homecontrol device display can be coordinated with images displayed on thetelevision or monitor. The information displayed on the home controldevice can identify a company or product and is coordinated with productplacement in an advertisement or program scene. Or it can be coordinatedwith subject matter of an advertisement or program scene. Theinformation can be displayed on the home control device as a usernavigates through content by fast forwarding, rewinding or skipping.

Other implementations may include a non-transitory computer readablestorage medium storing instructions executable by a processor to performa method as described above. Yet another implementation may include amethod including actions implemented by the processors of the device ofsystem as described above.

Implementation #23

Some implementations describe a method of delivering information to aconsumer, including: pairing a concierge device with a consumerelectronic (abbreviated CE) device via a two-way wireless RF connection;the concierge device receiving a data stream from the CE device anddisplaying information from the data stream on a display integral to theremote control.

This method and other implementations of the technology disclosed caneach optionally include one or more of the following features.

In some implementations, the concierge device relies on the CE device totranscode information from an MPEG transmission stream into a simplifiedconcierge device format.

As above, when the consumer device is a television or video image sourcecoupled to a monitor, the information can be displayed on the conciergedevice display is coordinated with images displayed on the television ormonitor. Other features of information displayed apply to this method aswell as the prior methods.

I claim:
 1. A non-transitory computer readable media, impressed withprogram instructions that, when executed on hardware, cause the hardwareto execute a method of supporting a consumer device using a home controldevice in communication with a cloud-based device, the method including:receiving a first signal from a particular device to be supported, via afirst two-way wireless, non-audio data channel used by the particulardevice to provide diagnostic related information to the home controldevice, wherein the first signal includes a device identifier thatidentifies the particular device; responsive to a single user actiondirected to the hardware, automatically determining which of a pluralityof service centers with which to initiate an interactive support sessionand initiating an interactive support session with the determinedservice center, wherein the determined service center supports theparticular device, including automatically transmitting to thedetermined service center, via a second two-way non-audio data channel,identifier information from which the determined service center canuniquely identify the particular device; conducting the interactivesupport session with the determined service center regarding theparticular device using the home control device to capture a user'svoice and to reproduce audio for the user to hear via a two-way audiochannel; and the interactive support session including relay of data viaa third two-way non-audio data channel between the particular device andthe service center simultaneously with the home control device capturingthe user's voice and to reproducing the audio for the user to hear. 2.The non-transitory computer readable media of claim 1, holding computerinstructions that when executed, cause performance of the method furtherincluding the home control device participating in a voice and datainteraction with an artificial intelligence system conducting theinteractive support session, the interactive support session includingthe home control device, the consumer device and the cloud-based device.3. The non-transitory computer readable media of claim 1, holdingcomputer instructions that when executed, cause performance of themethod further including: receiving a request from the service centervia the second two-way non-audio data channel for additional diagnosticinformation from the particular device; requesting and receiving fromthe particular device to be supported via the two-way wireless,non-audio data channel, additional responsive diagnostic information;and forwarding to the service center via the second two-way non-audiodata channel the additional responsive diagnostic information from theparticular device.
 4. The non-transitory computer readable media ofclaim 1, holding computer instructions that when executed, causeperformance of the method further including receiving from a userinterface of the hardware a user selection of the particular device tobe supported from a plurality of devices.
 5. A non-transitory computerreadable media, impressed with program instructions that, when executedon hardware, cause the hardware to execute a method of supporting aconsumer device using a home control device in communication with acloud-based device, the method including: responsive to a single useraction directed to the hardware, automatically determining which of aplurality of service centers with which to initiate an interactivesupport session for a particular device to be supported based on adevice identifier of the particular device; initiating an interactivesupport session with the determined service center, wherein thedetermined service center supports the particular device, includingautomatically transmitting to the determined service center, via a firsttwo-way non-audio data channel, identifier information from which thedetermined service center can uniquely identify the particular device;conducting the interactive support session with the determined servicecenter regarding the particular device using the home control device tocapture a user's voice and to reproduce audio for the user to hear via atwo-way audio channel; and the interactive support session includingrelay of data between the particular device and the service center via asecond two-way non-audio data channel simultaneously with the homecontrol devices capturing the user's voice and to reproducing the audiofor the user to hear.
 6. The non-transitory computer readable media ofclaim 5, further including program instructions that prompt thecloud-based device to participate in a voice and data interaction withan artificial intelligence system conducting an automated supportprotocol, the interaction including the home control device, theconsumer device and the cloud-based device.
 7. The non-transitorycomputer readable media of claim 5, holding computer instructions thatwhen executed, cause performance of the method further includingreceiving from a user interface of the hardware a user selection of theparticular device to be supported from a plurality of devices.
 8. Thenon-transitory computer readable media of claim 5, wherein the programinstructions run on hardware of a home control device that is physicallyincorporated in the consumer device.
 9. The non-transitory computerreadable media of claim 5, wherein the program instructions cause thehardware to execute the method further including sending controlcommands to the consumer device.
 10. The non-transitory computerreadable media of claim 5, further including program instructions thatprompt the cloud-based device to determine positively to engage anautomated support protocol based on the consumer device to be supportedand the user.
 11. The non-transitory computer readable media of claim 5,further including program instructions to cause the cloud-based deviceto determine to bypass an interactive voice recognition system based onthe determination of the consumer device.
 12. The non-transitorycomputer readable media of claim 5, further including programinstructions that prompt the cloud-based device to bypass an interactivevoice recognition system and to engage the user with a live person basedon the determination of the consumer device.